-- Logs begin at Wed 2022-08-24 09:13:30 UTC, end at Wed 2022-08-24 10:50:23 UTC. -- Aug 24 09:49:00.210152 dv-cinder-854270-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 09:49:00.220666 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 09:49:00.222937 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 09:49:00 2022] *** Aug 24 09:49:00.222937 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 09:49:00.222937 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 09:49:00.222937 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: nodename: dv-cinder-854270-2-iscsi Aug 24 09:49:00.222937 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: machine: x86_64 Aug 24 09:49:00.222937 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: clock source: unix Aug 24 09:49:00.222937 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: pcre jit disabled Aug 24 09:49:00.222937 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: detected number of CPU cores: 4 Aug 24 09:49:00.222937 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: current working directory: / Aug 24 09:49:00.222937 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: detected binary path: /usr/bin/uwsgi-core Aug 24 09:49:00.224184 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: your processes number limit is 47680 Aug 24 09:49:00.224184 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: your memory page size is 4096 bytes Aug 24 09:49:00.224184 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: detected max file descriptor number: 2048 Aug 24 09:49:00.224184 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: lock engine: pthread robust mutexes Aug 24 09:49:00.224184 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: thunder lock: enabled Aug 24 09:49:00.224184 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 09:49:00.224184 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 09:49:00.262836 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: Python main interpreter initialized at 0x563369a899c0 Aug 24 09:49:00.262836 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: python threads support enabled Aug 24 09:49:00.262836 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: your server socket listen backlog is limited to 100 connections Aug 24 09:49:00.262836 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: your mercy for graceful operations on workers is 90 seconds Aug 24 09:49:00.263178 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 09:49:00.263260 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: *** Operational MODE: preforking *** Aug 24 09:49:00.263512 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: *** uWSGI is running in multiple interpreter mode *** Aug 24 09:49:00.263512 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: spawned uWSGI master process (pid: 81452) Aug 24 09:49:00.263707 dv-cinder-854270-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 09:49:00.264703 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: spawned uWSGI worker 1 (pid: 81463, cores: 1) Aug 24 09:49:00.264703 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: spawned uWSGI worker 2 (pid: 81464, cores: 1) Aug 24 09:49:00.265370 dv-cinder-854270-2-iscsi devstack@keystone.service[81452]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 09:49:04.549392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:49:04.549392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: from cryptography.utils import int_from_bytes Aug 24 09:49:04.550351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:49:04.550351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: from cryptography.utils import int_from_bytes Aug 24 09:49:04.550505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:49:04.550505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: from cryptography.utils import int_from_bytes Aug 24 09:49:04.550921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:49:04.550921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: from cryptography.utils import int_from_bytes Aug 24 09:49:04.599904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 09:49:04.600119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 09:49:04.600314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] command line args: [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 09:49:04.600439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 09:49:04.600581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 09:49:04.600667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 09:49:04.600746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ================================================================================ {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 09:49:04.600825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] command line args: [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 09:49:04.600995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] admin_token = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.601081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 09:49:04.601238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] config_dir = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.601323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ================================================================================ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 09:49:04.601426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.601559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] admin_token = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.601636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] config_source = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.601769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] debug = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.601852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config_dir = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.601973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.602232 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.602232 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config_source = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.602353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] debug = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.602411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.602560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.602645 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.602735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.602802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.602922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.603001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.603166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.603252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.603360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_config_append = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.603440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.603536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.603638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.603718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_dir = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.603818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_config_append = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.603897 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_file = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.603997 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.604103 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.604174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_dir = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.604458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.604551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.604626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.604693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.604833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.604963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.605088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.605210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.605331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.605446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.605594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.605711 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.605822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.605942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.606064 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.606183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.606308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.606431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.606546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.606658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.606770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.606882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.607001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.607122 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.607226 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.607341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.607455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.607567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.607567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.607783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.86/identity {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.607898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.608016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] publish_errors = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.608167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.608292 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.86/identity {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.608414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.608533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] publish_errors = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.608645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.608764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.608878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.608996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.609108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.609238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.609360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] standard_threads = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.609475 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.609594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.609707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.609819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.609942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] standard_threads = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.610056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.610056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] use_journal = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.610274 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.610393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] use_json = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.610512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.610673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] use_stderr = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.610795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.610921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] use_syslog = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.611041 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_journal = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.611155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.611268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_json = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.611268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_stderr = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.611495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.611610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_syslog = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.611723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.611841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:49:04.611954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.612093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.612217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.612337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.612457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.612576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.612691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.612812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.612925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.613043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.613157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.613276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.613390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.613514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.613634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.613755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.613869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.613981 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.614093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.614093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.614297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.614418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.614544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.614658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.614772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.614900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.615021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.615134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.615246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.615358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.615469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.615628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.615866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.615979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.616120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.616319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.616436 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.616549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.616668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.616782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.616896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.617020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.617134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.617247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.617365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.617479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.617597 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.617719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.617833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.617953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.618067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.618180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.618292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.618412 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.618532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.618645 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.618757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.618870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.618989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.619112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.619226 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.619339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.619451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.619570 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.619683 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.619803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.619917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.620095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.620219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.620339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.620460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.620582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.620696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.620814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.620928 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.621047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.621161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.621281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.621396 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.621507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.621626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.621739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.621851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.622034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.622157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.622272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.622392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.622506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.622618 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.622743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.622858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.622858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.623068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.623181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.623294 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.623414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.623529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.623641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.623753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.623864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.623976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.624106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.624212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.624308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.624409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.624508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.624603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.624705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.624802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.624905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.625040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.625166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.625290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.625405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.625518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.625631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.625778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.625900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.626023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.626149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.626265 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.626380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.626492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.626605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.626726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.626847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.626974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.627095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.627219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.627335 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.627445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.627558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.627679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.627794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.627907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.628020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.628146 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.628272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.628387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.628508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.628622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.628735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.628854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.628967 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.629087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.629209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.629314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.629417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.629529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.629641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.629770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.629891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.630005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.630129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.630243 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.630356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.630478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.630592 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.630712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.630832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.630945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.631057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.631178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.631292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.631405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.631519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.631632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.631744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.631865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.631980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.632147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.632264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.632378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.632491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.632618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.632733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.632848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.632960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.632960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.633173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.633359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.633484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.633600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.633720 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.633834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.633957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.634082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.634208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.634323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.634441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.634555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.634670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.634783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.634904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.635020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.635133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.635247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.635360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.635473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.636387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.636512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.external = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.636628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.636752 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.636868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.636990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.637115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.637230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.637344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.637457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.password = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.637571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.637712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] auth.token = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.637846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.external = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.637964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.638087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.638203 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.638318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.638440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.638556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.638670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.638793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.password = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.638911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.639025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.token = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.639139 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.639253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.639368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.639491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.639607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.639728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.639843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.639959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.640088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.640217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.640336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.640451 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.640565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.640680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.640794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.640917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.641034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.641148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.641262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.641376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.641491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.641614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.641730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.641845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.641958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.642073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.642187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.642310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.642435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.642550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.642664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.642778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.642893 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.643019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.643136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.643250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.643365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.643365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.643578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.643701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.643817 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.643931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.644098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.644217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.644332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.644460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.644577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.644692 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.644806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.644923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.645046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.645163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.645278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.645392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.645506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.645620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.645743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.645858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.645973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.646088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.646240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.646358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.646482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.646606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.646722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.646836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.646951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.647066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.647189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.647306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.647427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.647543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.647665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.647780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.647780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.648085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.648202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.648318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.648433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.648547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.648674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.648791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.648905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.649020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.649134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.649248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.649372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.649496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.649612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.649726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.649840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.649955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.650078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.650194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.650307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.650423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.650536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.650650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.650773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.650888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.651003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.651117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.651328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.652016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.652197 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.652316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.652432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.652548 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.652663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.652785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.652909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.653026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.653141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.653256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.653377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.653493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.653618 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.653734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.653849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.653964 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.653964 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.654177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.654301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.654417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.654539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.654656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.654656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.654880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.655000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.655115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.655229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.655344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.655458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.655584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.655701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.655816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.655816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.656057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.656176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.656342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.656468 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.656584 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.656947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.657064 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.657179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.657301 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.657418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.657532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.657652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.657769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.657883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.658006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.658107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.658221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.658344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.658460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.658575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.658698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.658815 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.658929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.659044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.659158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.659273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.659395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.659511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.659626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.659626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.659840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.659954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.660161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.660281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.660396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.660510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.660625 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.660739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.660863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.660980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.661094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.661208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.661323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.661437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.661563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.661680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.661794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.661910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.662024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.662138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.662138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.662373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.662489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.662604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.662718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.662832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.662960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.663077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.663191 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.663306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.663421 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.663536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.663659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.663775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.663880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.663994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.664135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.664252 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.664375 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.664492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.664674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.664793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.664907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.665026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.665151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.665268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.665382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.665503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.665619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.665733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.665856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.665978 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.666101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.666217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.666369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.666488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.666612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.666728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.666868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.666988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.667103 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.667218 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.667341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.667458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.667572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.667687 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.667801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.667916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.668116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.668241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.668357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.668471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.668586 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.668700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.668828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.668946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.669061 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.669176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.669292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.669406 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.669529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.669646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.669761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.669875 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.669990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.670114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.670230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] role.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.670345 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.670460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] role.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.670460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.670675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.670798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.670916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.671031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.671145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.671259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.671375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.671566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.671686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.671800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.671915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.672029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.672164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.672292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.672412 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.672526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.672641 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.672755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.672870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.672992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.673109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.673223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.673338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.673454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.673568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.673705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.673823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.673823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.674038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.674152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.674266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.674389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.674506 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.674621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.674735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.674850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.674964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.675092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.675208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.675322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.675436 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.675551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.675665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.675789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.675905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.675905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.676176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.676293 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.676463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.676585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.676701 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.676815 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.676929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.677043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.677165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.677282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.677397 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.677512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.677626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.677740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.677740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.677965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.678080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.678194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.678308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.678428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.678552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.678668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.678783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.678897 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.679011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.679125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.679247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.679363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.679477 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.679590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.679703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.679818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.679940 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.680118 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.680237 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.680352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.680465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.680579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.680705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.680823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.680944 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.681059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.681173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.681288 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.681411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.681527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.681641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.681755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.681870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.681984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.682107 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.682229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.682345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.682478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.682597 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.connection = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.682720 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.682845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.682970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.683086 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.683201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.683315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.683429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.683543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.683666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.683782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.683896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.684011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.684148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.684264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.684390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.684507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.684622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.684736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.684850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.684965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.684965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.685195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.685319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.685443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.685560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.685675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.685790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.685905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.686029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.686145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.686260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.686374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.686525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.686644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.686768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.686885 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.687000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 09:49:04.687116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.687116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.687116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.687116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.687116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.687116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.687116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.687116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.687116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.688109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.688109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:49:04.688109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 09:49:04.714409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.714638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.714860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.716079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.716309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.716540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.723367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.724977 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.751409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.751711 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.753045 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.753287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.807186 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.807334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.856968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.857107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.857202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:04.857304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.015361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.015591 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.015811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.015897 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.016036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.016147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.016286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.016370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.016508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.016605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.016722 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.016938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.039905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.040160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.040281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.040401 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.040548 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.040629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.040704 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.040842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.040935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.041065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.041148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.041285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.041366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.041498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.052529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.052873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.065946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.066130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.066384 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.066570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.066656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.066794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.066876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.067011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.067103 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.067226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.067308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.067436 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.067647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.067859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:49:05.084642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.084858 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:49:05.085206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.085288 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.085411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:49:05.085509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:49:05.085691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.085775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.085945 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:49:05.086026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.086224 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.086469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.086552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.086742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.086976 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.087239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.096183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.096472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.096590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.096836 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.096974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.097134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.097257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.097479 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.104655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.105209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.107529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.107818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.108135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.108221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.108423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.108505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.108709 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.108794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.108973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.109056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.109247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.109338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.109523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.109605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.109802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.109905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.110085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.110208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.110439 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.110519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.110747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.111026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.133403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.134104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.134853 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.135138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.135573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.136440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.140374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.140685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.140944 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.141170 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.141465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.141756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.147270 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.147559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.148151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.148449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.151664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.151957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.152268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.152556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.152637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.152837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.152920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.153116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.153197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.153388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.153479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.153661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.153742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.153983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.154262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.154538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.170972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.171253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.172152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.172444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.175135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.176349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.178353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.178647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.179648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.179953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.183359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.184749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.186502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.187887 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.189737 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.190040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.190328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.190625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.190910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.191167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.191260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.191475 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.191778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.192098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.192398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.192696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.207458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.207740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.208008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.208296 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.209037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.209339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.209615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.209888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.215165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.216724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.216854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.217002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.218391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.218677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.219636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.219924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.220240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.220531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.220808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.221368 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.221667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.222013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.222392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.222688 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.232077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.232356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.232640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.232921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.233206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.233486 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.233773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.234411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.234702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.234987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.235275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.235566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.235851 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.236165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.249613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.252143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.252647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.255193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.255533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.256998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.258133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.258462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.258744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.259033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.259591 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.261657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.261947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.262246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.266069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.267518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.268781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.270264 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.270454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.270748 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.271036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.271321 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.273189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.273487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.273789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.274082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.279270 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.279562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.279844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.282381 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.282680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.282971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.287006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.287285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.287569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.287847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.288155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.288429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.288710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.288987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.289265 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.289534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.289817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.289942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:49:05.290080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.290236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.290360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.290527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.290609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.290812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.291105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.291386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.291687 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.292495 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.292781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.293067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.293358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.293635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.293914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.294182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:49:05.325443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563369a899c0 pid: 81464 (default app) Aug 24 09:49:05.328703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563369a899c0 pid: 81463 (default app) Aug 24 09:49:05.328797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88278424-6486-4ff3-a661-985d153b2fab None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:05.329084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88278424-6486-4ff3-a661-985d153b2fab None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:05.329356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88278424-6486-4ff3-a661-985d153b2fab None None] PATH_INFO: `/v3` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:05.333516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1/1] 10.222.0.86 () {50 vars in 791 bytes} [Wed Aug 24 09:49:05 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:11.877748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90990192-17e4-4603-ba27-3285d7eeea64 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:11.878254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90990192-17e4-4603-ba27-3285d7eeea64 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:11.878336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90990192-17e4-4603-ba27-3285d7eeea64 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:11.882628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1/2] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:11.890294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e508cf0-2f44-4916-a4c1-5ff7849e7f7d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:11.890753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e508cf0-2f44-4916-a4c1-5ff7849e7f7d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:11.890835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e508cf0-2f44-4916-a4c1-5ff7849e7f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:11.960839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: INFO dbcounter [None req-2e508cf0-2f44-4916-a4c1-5ff7849e7f7d None None] Registered counter for database keystone Aug 24 09:49:11.979623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG oslo_db.sqlalchemy.engines [None req-2e508cf0-2f44-4916-a4c1-5ff7849e7f7d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81464) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:49:11.986006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writer thread running {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:49:12.166356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG passlib.handlers.bcrypt [None req-2e508cf0-2f44-4916-a4c1-5ff7849e7f7d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81464) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 09:49:12.166698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG passlib.handlers.bcrypt [None req-2e508cf0-2f44-4916-a4c1-5ff7849e7f7d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81464) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 09:49:12.223974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2e508cf0-2f44-4916-a4c1-5ff7849e7f7d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:12.225612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2e508cf0-2f44-4916-a4c1-5ff7849e7f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:12.354104 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 2/3] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 09:49:12.814675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90990192-17e4-4603-ba27-3285d7eeea64 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:12.882650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: INFO dbcounter [None req-90990192-17e4-4603-ba27-3285d7eeea64 None None] Registered counter for database keystone Aug 24 09:49:12.905142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG oslo_db.sqlalchemy.engines [None req-90990192-17e4-4603-ba27-3285d7eeea64 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81463) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:49:12.911573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writer thread running {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:49:13.030128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90990192-17e4-4603-ba27-3285d7eeea64 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:13.034561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1472d6-7561-439e-93df-3b782add3bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:13.035176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1472d6-7561-439e-93df-3b782add3bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:13.035465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1472d6-7561-439e-93df-3b782add3bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:13.035747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1472d6-7561-439e-93df-3b782add3bb9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:13.043919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1472d6-7561-439e-93df-3b782add3bb9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:13.095497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6e1472d6-7561-439e-93df-3b782add3bb9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 09:49:13.095843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1472d6-7561-439e-93df-3b782add3bb9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:13.105364 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 2/4] 10.222.0.86 () {58 vars in 1324 bytes} [Wed Aug 24 09:49:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 09:49:13.111370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e508cf0-2f44-4916-a4c1-5ff7849e7f7d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:13.125644 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e508cf0-2f44-4916-a4c1-5ff7849e7f7d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:13.129968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd59d5-fceb-4360-af55-c5c7bcbbe568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:13.131066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd59d5-fceb-4360-af55-c5c7bcbbe568 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:13.131351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd59d5-fceb-4360-af55-c5c7bcbbe568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:13.131623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd59d5-fceb-4360-af55-c5c7bcbbe568 admin admin] PATH_INFO: `/v3/projects/f71a167399734714a60b777eea4e2ecc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:13.133259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd59d5-fceb-4360-af55-c5c7bcbbe568 admin admin] RBAC: Authorizing `identity:get_project(project_id=f71a167399734714a60b777eea4e2ecc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:13.187060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c0dd59d5-fceb-4360-af55-c5c7bcbbe568 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 09:49:13.187400 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd59d5-fceb-4360-af55-c5c7bcbbe568 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:13.192327 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 3/5] 10.222.0.86 () {56 vars in 1207 bytes} [Wed Aug 24 09:49:13 2022] GET /identity/v3/projects/f71a167399734714a60b777eea4e2ecc => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:15.110099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe28ae-6638-4e2d-94bd-379f6c9f6d3f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:15.110616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe28ae-6638-4e2d-94bd-379f6c9f6d3f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:15.110616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe28ae-6638-4e2d-94bd-379f6c9f6d3f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:15.114708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 3/6] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:15.120204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338341fa-3720-4a47-aa05-77faf12977d0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:15.120627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338341fa-3720-4a47-aa05-77faf12977d0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:15.120765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338341fa-3720-4a47-aa05-77faf12977d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:15.141449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-338341fa-3720-4a47-aa05-77faf12977d0 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:15.142967 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-338341fa-3720-4a47-aa05-77faf12977d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:15.150961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 4/7] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 09:49:15.415492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe28ae-6638-4e2d-94bd-379f6c9f6d3f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:15.427436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe28ae-6638-4e2d-94bd-379f6c9f6d3f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:15.431561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1dfc3b-214b-4f0e-802d-11bf1e4c598b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:15.432184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dfc3b-214b-4f0e-802d-11bf1e4c598b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:15.432463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dfc3b-214b-4f0e-802d-11bf1e4c598b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:15.432733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dfc3b-214b-4f0e-802d-11bf1e4c598b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:15.440792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1dfc3b-214b-4f0e-802d-11bf1e4c598b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:15.442570 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1dfc3b-214b-4f0e-802d-11bf1e4c598b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:15.451906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 4/8] 10.222.0.86 () {58 vars in 1324 bytes} [Wed Aug 24 09:49:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 09:49:15.457168 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338341fa-3720-4a47-aa05-77faf12977d0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:15.466238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338341fa-3720-4a47-aa05-77faf12977d0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:15.470601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e054dc-2136-4615-bcf1-b9bcb723d1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:15.471224 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e054dc-2136-4615-bcf1-b9bcb723d1dc admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:15.471508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e054dc-2136-4615-bcf1-b9bcb723d1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:15.471782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e054dc-2136-4615-bcf1-b9bcb723d1dc admin admin] PATH_INFO: `/v3/users/9757335c28f74ea5a8fbdedc0ae7682a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:15.473116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e054dc-2136-4615-bcf1-b9bcb723d1dc admin admin] RBAC: Authorizing `identity:get_user(user_id=9757335c28f74ea5a8fbdedc0ae7682a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:15.474958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e054dc-2136-4615-bcf1-b9bcb723d1dc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:15.476657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 5/9] 10.222.0.86 () {56 vars in 1198 bytes} [Wed Aug 24 09:49:15 2022] GET /identity/v3/users/9757335c28f74ea5a8fbdedc0ae7682a => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:17.429654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81cc86-e1d4-433c-832c-cf3ce1bd0636 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:17.430208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81cc86-e1d4-433c-832c-cf3ce1bd0636 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:17.430208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81cc86-e1d4-433c-832c-cf3ce1bd0636 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:17.431558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 5/10] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:49:17.436873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab79db0-40d7-4d5f-99ad-27e1bbcd677c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:17.437316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab79db0-40d7-4d5f-99ad-27e1bbcd677c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:17.437412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab79db0-40d7-4d5f-99ad-27e1bbcd677c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:17.455534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fab79db0-40d7-4d5f-99ad-27e1bbcd677c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:17.456747 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fab79db0-40d7-4d5f-99ad-27e1bbcd677c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:17.459300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e10ca6-4bde-4961-ae82-51d3d104ba47 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:17.459592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e10ca6-4bde-4961-ae82-51d3d104ba47 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:17.459874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e10ca6-4bde-4961-ae82-51d3d104ba47 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:17.461230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 6/11] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:49:17.467093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba5d2a-9034-46d3-995b-4516b49e30d3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:17.467383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba5d2a-9034-46d3-995b-4516b49e30d3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:17.467699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba5d2a-9034-46d3-995b-4516b49e30d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:17.509157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG passlib.handlers.bcrypt [None req-95ba5d2a-9034-46d3-995b-4516b49e30d3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81463) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 09:49:17.509581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG passlib.handlers.bcrypt [None req-95ba5d2a-9034-46d3-995b-4516b49e30d3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81463) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 09:49:17.537561 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 6/12] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:17.555762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-95ba5d2a-9034-46d3-995b-4516b49e30d3 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:17.557387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-95ba5d2a-9034-46d3-995b-4516b49e30d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:17.654905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 7/13] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:17.804525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab79db0-40d7-4d5f-99ad-27e1bbcd677c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:17.829144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:17.909636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab79db0-40d7-4d5f-99ad-27e1bbcd677c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:17.926949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba5d2a-9034-46d3-995b-4516b49e30d3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:17.943155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7860f-f3e3-451f-aef3-3d559685f575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:17.943796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7860f-f3e3-451f-aef3-3d559685f575 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:17.944128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7860f-f3e3-451f-aef3-3d559685f575 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:17.944409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7860f-f3e3-451f-aef3-3d559685f575 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:17.956855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7860f-f3e3-451f-aef3-3d559685f575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-93e7860f-f3e3-451f-aef3-3d559685f575 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:17.965351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:17.966432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:17.967263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 7/14] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:17.972235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7860f-f3e3-451f-aef3-3d559685f575 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:18.039937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba5d2a-9034-46d3-995b-4516b49e30d3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:18.042277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:18.067256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:18.079957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a2a23-e54e-4c7e-a779-e957c2163e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:18.080841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a2a23-e54e-4c7e-a779-e957c2163e7c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:18.081121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a2a23-e54e-4c7e-a779-e957c2163e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:18.081391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a2a23-e54e-4c7e-a779-e957c2163e7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:18.087685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7860f-f3e3-451f-aef3-3d559685f575 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:18.121183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f946-d7ec-4aea-aacf-4e304dc6e4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:18.121784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f946-d7ec-4aea-aacf-4e304dc6e4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:18.122679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f946-d7ec-4aea-aacf-4e304dc6e4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:18.122962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f946-d7ec-4aea-aacf-4e304dc6e4a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:18.123630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5f946-d7ec-4aea-aacf-4e304dc6e4a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:18.123934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5f946-d7ec-4aea-aacf-4e304dc6e4a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:18.125371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5f946-d7ec-4aea-aacf-4e304dc6e4a6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:18.136926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 8/15] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:18.142313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f946-d7ec-4aea-aacf-4e304dc6e4a6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:18.148422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a2a23-e54e-4c7e-a779-e957c2163e7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:18.149953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a2a23-e54e-4c7e-a779-e957c2163e7c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:18.241868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f946-d7ec-4aea-aacf-4e304dc6e4a6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:18.252138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 8/16] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:49:18.257668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a2a23-e54e-4c7e-a779-e957c2163e7c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:18.260382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:18.279502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd895a3-3960-49e0-87e8-ca4ff4b859c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:18.280168 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd895a3-3960-49e0-87e8-ca4ff4b859c9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:18.280452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd895a3-3960-49e0-87e8-ca4ff4b859c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:18.280731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd895a3-3960-49e0-87e8-ca4ff4b859c9 None admin] PATH_INFO: `/v3/users/9757335c28f74ea5a8fbdedc0ae7682a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:18.281873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd895a3-3960-49e0-87e8-ca4ff4b859c9 None admin] RBAC: Authorizing `identity:get_user(user_id=9757335c28f74ea5a8fbdedc0ae7682a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:18.283358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd895a3-3960-49e0-87e8-ca4ff4b859c9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:18.285232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 9/17] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:18 2022] GET /identity/v3/users/9757335c28f74ea5a8fbdedc0ae7682a => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:18.290779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd895a3-3960-49e0-87e8-ca4ff4b859c9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:18.305510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:18.357339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a2a23-e54e-4c7e-a779-e957c2163e7c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:18.392012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd895a3-3960-49e0-87e8-ca4ff4b859c9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:18.395407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17755586-0005-47b6-a184-b4cea1289dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:18.396305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17755586-0005-47b6-a184-b4cea1289dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:18.396593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17755586-0005-47b6-a184-b4cea1289dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:18.396841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17755586-0005-47b6-a184-b4cea1289dc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:18.415753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17755586-0005-47b6-a184-b4cea1289dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:18.417259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17755586-0005-47b6-a184-b4cea1289dc3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:18.426415 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b16ca-b19a-4d9c-abb0-a54fe58f90f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:18.427117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b16ca-b19a-4d9c-abb0-a54fe58f90f8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:18.427401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b16ca-b19a-4d9c-abb0-a54fe58f90f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:18.427675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b16ca-b19a-4d9c-abb0-a54fe58f90f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:18.428824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b16ca-b19a-4d9c-abb0-a54fe58f90f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:18.430270 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b16ca-b19a-4d9c-abb0-a54fe58f90f8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:18.431823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 10/18] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-17755586-0005-47b6-a184-b4cea1289dc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:49:18.436003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:18.437229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:49:18.438060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:49:18.438060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:49:18.438060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:18.438060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 9/19] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:49:18.438265 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b16ca-b19a-4d9c-abb0-a54fe58f90f8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:18.442928 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17755586-0005-47b6-a184-b4cea1289dc3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:18.502998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:18.518841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:18.538402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b16ca-b19a-4d9c-abb0-a54fe58f90f8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:18.548390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17755586-0005-47b6-a184-b4cea1289dc3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:18.571922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3077a1-e270-4588-a4fd-53761510c711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:18.572549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3077a1-e270-4588-a4fd-53761510c711 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:18.572838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3077a1-e270-4588-a4fd-53761510c711 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:18.573097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3077a1-e270-4588-a4fd-53761510c711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:18.573820 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3077a1-e270-4588-a4fd-53761510c711 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.domain.id=default, user.id=9757335c28f74ea5a8fbdedc0ae7682a) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:18.574125 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3077a1-e270-4588-a4fd-53761510c711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:18.575621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3077a1-e270-4588-a4fd-53761510c711 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:18.582291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 11/20] 10.222.0.86 () {56 vars in 1322 bytes} [Wed Aug 24 09:49:18 2022] GET /identity/v3/role_assignments?user.id=9757335c28f74ea5a8fbdedc0ae7682a&scope.domain.id=default&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:18.582408 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197845b-c1e4-4445-84ea-b65c60ee2e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:18.582918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197845b-c1e4-4445-84ea-b65c60ee2e20 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:18.583241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197845b-c1e4-4445-84ea-b65c60ee2e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:18.583575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197845b-c1e4-4445-84ea-b65c60ee2e20 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:18.584322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197845b-c1e4-4445-84ea-b65c60ee2e20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:18.584653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197845b-c1e4-4445-84ea-b65c60ee2e20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:18.586169 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197845b-c1e4-4445-84ea-b65c60ee2e20 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:18.608663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 10/21] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:20.938089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9712d-917a-4714-8704-fa0d0e37513c None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:20.938614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9712d-917a-4714-8704-fa0d0e37513c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:20.938614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9712d-917a-4714-8704-fa0d0e37513c None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:20.940267 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 12/22] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:20.945541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b41938-7144-46fa-8cb6-a15545950894 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:20.946382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b41938-7144-46fa-8cb6-a15545950894 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:20.946857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b41938-7144-46fa-8cb6-a15545950894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:20.977025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-73b41938-7144-46fa-8cb6-a15545950894 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:20.978702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-73b41938-7144-46fa-8cb6-a15545950894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:21.069494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 11/23] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:21.088502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3893d7a-f9c2-41d5-bcd8-b04f2b3c35c8 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:21.089090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3893d7a-f9c2-41d5-bcd8-b04f2b3c35c8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:21.089620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3893d7a-f9c2-41d5-bcd8-b04f2b3c35c8 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:21.092141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 13/24] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:21.097210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b85c5-50ae-47c0-9b12-ac462cd4ef54 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:21.097495 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b85c5-50ae-47c0-9b12-ac462cd4ef54 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:21.097766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b85c5-50ae-47c0-9b12-ac462cd4ef54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:21.123136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-776b85c5-50ae-47c0-9b12-ac462cd4ef54 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:21.124350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-776b85c5-50ae-47c0-9b12-ac462cd4ef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:21.124692 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab522b12-5a8c-4b52-a119-e19bc4f37046 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:21.125180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab522b12-5a8c-4b52-a119-e19bc4f37046 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:21.125660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab522b12-5a8c-4b52-a119-e19bc4f37046 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:21.127198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 14/25] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:49:21.132177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309343-1804-403c-a14e-2f7f70efaaf6 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:21.132704 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309343-1804-403c-a14e-2f7f70efaaf6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:21.133190 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309343-1804-403c-a14e-2f7f70efaaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:21.166697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d5309343-1804-403c-a14e-2f7f70efaaf6 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:21.168430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d5309343-1804-403c-a14e-2f7f70efaaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:21.186285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:21.233389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 12/26] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:21.312530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd52b5-eb2c-4def-96bd-d02ee5856e58 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:21.312866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd52b5-eb2c-4def-96bd-d02ee5856e58 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:21.313063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd52b5-eb2c-4def-96bd-d02ee5856e58 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:21.314404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 13/27] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:49:21.320262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3cfb8-c893-49d6-9ac2-425fd3737c4f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:21.320502 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3cfb8-c893-49d6-9ac2-425fd3737c4f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:21.320748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3cfb8-c893-49d6-9ac2-425fd3737c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:21.321823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 15/28] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:21.349122 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cfc3cfb8-c893-49d6-9ac2-425fd3737c4f None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:21.350337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cfc3cfb8-c893-49d6-9ac2-425fd3737c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:21.448530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 14/29] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:21.486481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5309343-1804-403c-a14e-2f7f70efaaf6 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:21.583797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:21.619796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5309343-1804-403c-a14e-2f7f70efaaf6 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:21.667194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0b598-d906-4e8d-93d4-6e6890e4e12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:21.667828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0b598-d906-4e8d-93d4-6e6890e4e12f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:21.668135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0b598-d906-4e8d-93d4-6e6890e4e12f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:21.668417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0b598-d906-4e8d-93d4-6e6890e4e12f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:21.680981 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3cfb8-c893-49d6-9ac2-425fd3737c4f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:21.681942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0b598-d906-4e8d-93d4-6e6890e4e12f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:21.683331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0b598-d906-4e8d-93d4-6e6890e4e12f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a9e0b598-d906-4e8d-93d4-6e6890e4e12f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:49:21.706917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:21.707802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:49:21.708633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:49:21.708633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:49:21.708633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:21.708633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 16/30] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:21.714095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0b598-d906-4e8d-93d4-6e6890e4e12f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:21.839929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0b598-d906-4e8d-93d4-6e6890e4e12f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:21.867143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:21.881689 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae09e09-c252-4271-bc26-80c0ba8c7a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:21.882309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09e09-c252-4271-bc26-80c0ba8c7a23 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:21.882592 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09e09-c252-4271-bc26-80c0ba8c7a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:21.882864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09e09-c252-4271-bc26-80c0ba8c7a23 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:21.883522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae09e09-c252-4271-bc26-80c0ba8c7a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:21.883828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae09e09-c252-4271-bc26-80c0ba8c7a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:21.885247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae09e09-c252-4271-bc26-80c0ba8c7a23 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:21.918790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3cfb8-c893-49d6-9ac2-425fd3737c4f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:21.923894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 17/31] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:21.928707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:21.930293 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae09e09-c252-4271-bc26-80c0ba8c7a23 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:21.969748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f2b0a-a139-4153-a350-5500c1708af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:21.970383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f2b0a-a139-4153-a350-5500c1708af2 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:21.970669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f2b0a-a139-4153-a350-5500c1708af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:21.970947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f2b0a-a139-4153-a350-5500c1708af2 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:21.971632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f2b0a-a139-4153-a350-5500c1708af2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:21.973193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f2b0a-a139-4153-a350-5500c1708af2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:22.047271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae09e09-c252-4271-bc26-80c0ba8c7a23 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:22.084098 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80227e-7639-4808-9c80-75ecbc6fbb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:22.084849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80227e-7639-4808-9c80-75ecbc6fbb01 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:22.085227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80227e-7639-4808-9c80-75ecbc6fbb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:22.085600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80227e-7639-4808-9c80-75ecbc6fbb01 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:22.086414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80227e-7639-4808-9c80-75ecbc6fbb01 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:22.088126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80227e-7639-4808-9c80-75ecbc6fbb01 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:22.124296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 15/32] 10.222.0.86 () {60 vars in 1131 bytes} [Wed Aug 24 09:49:21 2022] POST /identity/v3/roles => generated 219 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:49:22.126390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f2b0a-a139-4153-a350-5500c1708af2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:22.244507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 18/33] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:49:21 2022] POST /identity/v3/projects => generated 301 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:49:22.245549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80227e-7639-4808-9c80-75ecbc6fbb01 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:22.252077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f2b0a-a139-4153-a350-5500c1708af2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:22.301396 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:22.339008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d0aae-e3ae-409d-bccf-0f89b02680ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:22.340014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d0aae-e3ae-409d-bccf-0f89b02680ad None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:22.340522 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d0aae-e3ae-409d-bccf-0f89b02680ad None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:22.340979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d0aae-e3ae-409d-bccf-0f89b02680ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:22.380308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80227e-7639-4808-9c80-75ecbc6fbb01 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:22.384824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d0aae-e3ae-409d-bccf-0f89b02680ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-878d0aae-e3ae-409d-bccf-0f89b02680ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:22.391901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:22.392865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:22.393623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:22.396076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 16/34] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:49:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:22.420977 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d0aae-e3ae-409d-bccf-0f89b02680ad None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:22.428611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a7214-6e2e-4561-a43a-bfdb10fa850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:22.429392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a7214-6e2e-4561-a43a-bfdb10fa850b None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:22.429766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a7214-6e2e-4561-a43a-bfdb10fa850b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:22.430110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a7214-6e2e-4561-a43a-bfdb10fa850b None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:22.430863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a7214-6e2e-4561-a43a-bfdb10fa850b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:22.432572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a7214-6e2e-4561-a43a-bfdb10fa850b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:22.456433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 19/35] 10.222.0.86 () {60 vars in 1131 bytes} [Wed Aug 24 09:49:22 2022] POST /identity/v3/roles => generated 225 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:49:22.458202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bfa6d-628a-47b5-8f61-e8b65145bb86 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:22.458697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bfa6d-628a-47b5-8f61-e8b65145bb86 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:22.459212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bfa6d-628a-47b5-8f61-e8b65145bb86 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:22.461039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 20/36] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:22.467154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972fe37-1263-4272-95c2-977650a704c3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:22.467779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972fe37-1263-4272-95c2-977650a704c3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:22.468334 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972fe37-1263-4272-95c2-977650a704c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:22.470099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:22.493567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f972fe37-1263-4272-95c2-977650a704c3 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:22.495598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f972fe37-1263-4272-95c2-977650a704c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:22.577896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d0aae-e3ae-409d-bccf-0f89b02680ad None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:22.594520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 21/37] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:22.621735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a371a7-7e9c-4b05-8d6c-b4345b30c1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:22.622735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a371a7-7e9c-4b05-8d6c-b4345b30c1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:22.623213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a371a7-7e9c-4b05-8d6c-b4345b30c1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:22.623685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a371a7-7e9c-4b05-8d6c-b4345b30c1a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:22.624561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a371a7-7e9c-4b05-8d6c-b4345b30c1a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:22.625009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a371a7-7e9c-4b05-8d6c-b4345b30c1a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:22.626800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a371a7-7e9c-4b05-8d6c-b4345b30c1a9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:22.627429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ab590-31ec-476e-83df-4736a29fa0a2 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:22.627827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ab590-31ec-476e-83df-4736a29fa0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:22.628329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ab590-31ec-476e-83df-4736a29fa0a2 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:22.630883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 22/38] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:49:22.636686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2dfa6-392c-4b01-806e-ff296b005dbf None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:22.637114 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2dfa6-392c-4b01-806e-ff296b005dbf None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:22.637519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2dfa6-392c-4b01-806e-ff296b005dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:22.650666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 17/39] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:22.658493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a371a7-7e9c-4b05-8d6c-b4345b30c1a9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:22.677460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fbe2dfa6-392c-4b01-806e-ff296b005dbf None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:22.678740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fbe2dfa6-392c-4b01-806e-ff296b005dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:22.711926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:22.771214 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 23/40] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:22.794922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:22.810668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a371a7-7e9c-4b05-8d6c-b4345b30c1a9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:22.846368 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6730eb0-0a01-42ac-9ea3-4d12a789f5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:22.847054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6730eb0-0a01-42ac-9ea3-4d12a789f5aa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:22.847343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6730eb0-0a01-42ac-9ea3-4d12a789f5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:22.847618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6730eb0-0a01-42ac-9ea3-4d12a789f5aa None admin] PATH_INFO: `/v3/users/9757335c28f74ea5a8fbdedc0ae7682a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:22.849351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6730eb0-0a01-42ac-9ea3-4d12a789f5aa None admin] RBAC: Authorizing `identity:get_user(user_id=9757335c28f74ea5a8fbdedc0ae7682a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:22.850798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6730eb0-0a01-42ac-9ea3-4d12a789f5aa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:22.852593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 18/41] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:22 2022] GET /identity/v3/users/9757335c28f74ea5a8fbdedc0ae7682a => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:22.857990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2dfa6-392c-4b01-806e-ff296b005dbf None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:22.917897 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6730eb0-0a01-42ac-9ea3-4d12a789f5aa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:22.961599 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2dfa6-392c-4b01-806e-ff296b005dbf None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:22.999945 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bdf35-b8fb-43e9-9ea5-4bd4b53af11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:23.000597 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bdf35-b8fb-43e9-9ea5-4bd4b53af11f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:23.000879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bdf35-b8fb-43e9-9ea5-4bd4b53af11f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:23.001262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bdf35-b8fb-43e9-9ea5-4bd4b53af11f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:23.002457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bdf35-b8fb-43e9-9ea5-4bd4b53af11f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:23.003909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bdf35-b8fb-43e9-9ea5-4bd4b53af11f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:23.005915 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 24/42] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:23.011347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bdf35-b8fb-43e9-9ea5-4bd4b53af11f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:23.021238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6730eb0-0a01-42ac-9ea3-4d12a789f5aa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:23.043511 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:23.058499 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a6fd8-3c11-4514-a21a-29a6bca37a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:23.059125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a6fd8-3c11-4514-a21a-29a6bca37a53 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:23.059407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a6fd8-3c11-4514-a21a-29a6bca37a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:23.059684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a6fd8-3c11-4514-a21a-29a6bca37a53 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:23.060923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a6fd8-3c11-4514-a21a-29a6bca37a53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:23.062356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a6fd8-3c11-4514-a21a-29a6bca37a53 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:23.080388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 19/43] 10.222.0.86 () {60 vars in 1131 bytes} [Wed Aug 24 09:49:22 2022] POST /identity/v3/roles => generated 223 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:49:23.081269 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a6fd8-3c11-4514-a21a-29a6bca37a53 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:23.105473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:23.211887 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a6fd8-3c11-4514-a21a-29a6bca37a53 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:23.238765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bdf35-b8fb-43e9-9ea5-4bd4b53af11f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:23.254003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863a2c8-3ba8-4e91-82ca-b64edd4e9c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:23.254650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863a2c8-3ba8-4e91-82ca-b64edd4e9c26 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:23.254958 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863a2c8-3ba8-4e91-82ca-b64edd4e9c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:23.255242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863a2c8-3ba8-4e91-82ca-b64edd4e9c26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:23.258095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863a2c8-3ba8-4e91-82ca-b64edd4e9c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:23.259587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863a2c8-3ba8-4e91-82ca-b64edd4e9c26 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:23.261549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 20/44] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:23.267819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863a2c8-3ba8-4e91-82ca-b64edd4e9c26 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:23.281306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b63dbc-4a23-42a4-8523-220b5d29c804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:23.282016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63dbc-4a23-42a4-8523-220b5d29c804 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:23.282307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63dbc-4a23-42a4-8523-220b5d29c804 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:23.282593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63dbc-4a23-42a4-8523-220b5d29c804 None admin] PATH_INFO: `/v3/domains/default/users/9757335c28f74ea5a8fbdedc0ae7682a/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:23.284608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b63dbc-4a23-42a4-8523-220b5d29c804 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9757335c28f74ea5a8fbdedc0ae7682a, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:23.286352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b63dbc-4a23-42a4-8523-220b5d29c804 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:23.301780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 25/45] 10.222.0.86 () {58 vars in 1361 bytes} [Wed Aug 24 09:49:23 2022] PUT /identity/v3/domains/default/users/9757335c28f74ea5a8fbdedc0ae7682a/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:49:23.362144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:23.380534 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863a2c8-3ba8-4e91-82ca-b64edd4e9c26 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:23.417370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f325b4d-6989-4cf5-a127-5010e85bf437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:23.418004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f325b4d-6989-4cf5-a127-5010e85bf437 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:23.418289 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f325b4d-6989-4cf5-a127-5010e85bf437 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:23.418565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f325b4d-6989-4cf5-a127-5010e85bf437 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:23.419242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f325b4d-6989-4cf5-a127-5010e85bf437 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:23.421286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f325b4d-6989-4cf5-a127-5010e85bf437 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:23.454533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 21/46] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:49:23 2022] POST /identity/v3/projects => generated 312 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:49:25.202567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04792acd-04b4-47dd-a130-079ca42635ae None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:25.203122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04792acd-04b4-47dd-a130-079ca42635ae None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:25.203208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04792acd-04b4-47dd-a130-079ca42635ae None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:25.205082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 26/47] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:25.210257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb72911-6e9f-4bde-8544-dc5e948716c5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:25.210649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb72911-6e9f-4bde-8544-dc5e948716c5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:25.210821 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb72911-6e9f-4bde-8544-dc5e948716c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:25.229210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7eb72911-6e9f-4bde-8544-dc5e948716c5 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:25.230709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7eb72911-6e9f-4bde-8544-dc5e948716c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:25.299225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 22/48] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:25.337466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ad406-a58a-43f2-88af-5321f4f4c553 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:25.337761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ad406-a58a-43f2-88af-5321f4f4c553 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:25.338046 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ad406-a58a-43f2-88af-5321f4f4c553 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:25.339415 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 27/49] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:25.345093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177021-cba4-4821-bf1a-68f58b01b34d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:25.345389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177021-cba4-4821-bf1a-68f58b01b34d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:25.345674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177021-cba4-4821-bf1a-68f58b01b34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:25.363986 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-10177021-cba4-4821-bf1a-68f58b01b34d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:25.365316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-10177021-cba4-4821-bf1a-68f58b01b34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:25.424837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:25.438603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 23/50] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:25.567017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ad406-a58a-43f2-88af-5321f4f4c553 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:25.650931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:25.668363 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ad406-a58a-43f2-88af-5321f4f4c553 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:25.699051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10177021-cba4-4821-bf1a-68f58b01b34d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:25.706121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0513d-3a5e-4d41-ac19-db5e8d0c3939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:25.706764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0513d-3a5e-4d41-ac19-db5e8d0c3939 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:25.707047 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0513d-3a5e-4d41-ac19-db5e8d0c3939 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:25.707321 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0513d-3a5e-4d41-ac19-db5e8d0c3939 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:25.718859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0513d-3a5e-4d41-ac19-db5e8d0c3939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-93f0513d-3a5e-4d41-ac19-db5e8d0c3939 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:25.723543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:25.724413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:25.725490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 28/51] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:49:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:25.730463 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0513d-3a5e-4d41-ac19-db5e8d0c3939 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:25.803073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10177021-cba4-4821-bf1a-68f58b01b34d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:25.836339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0513d-3a5e-4d41-ac19-db5e8d0c3939 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:25.836903 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdf38e-cd9c-43a6-8971-f5c448347900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:25.837538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdf38e-cd9c-43a6-8971-f5c448347900 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:25.837822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdf38e-cd9c-43a6-8971-f5c448347900 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:25.838098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdf38e-cd9c-43a6-8971-f5c448347900 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:25.839213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebdf38e-cd9c-43a6-8971-f5c448347900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:25.840715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebdf38e-cd9c-43a6-8971-f5c448347900 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:25.842911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 24/52] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:25.848489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdf38e-cd9c-43a6-8971-f5c448347900 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:25.869473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e5781-3835-40bc-8cc4-b66cee0e18cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:25.870072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5781-3835-40bc-8cc4-b66cee0e18cf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:25.870374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5781-3835-40bc-8cc4-b66cee0e18cf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:25.870626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5781-3835-40bc-8cc4-b66cee0e18cf None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:25.871286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e5781-3835-40bc-8cc4-b66cee0e18cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:25.871597 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e5781-3835-40bc-8cc4-b66cee0e18cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:25.873038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e5781-3835-40bc-8cc4-b66cee0e18cf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:25.881303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 29/53] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:25.886904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e5781-3835-40bc-8cc4-b66cee0e18cf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:25.897577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:25.910960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:25.949481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdf38e-cd9c-43a6-8971-f5c448347900 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:25.984017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebafddd-7533-441f-bf78-2778acc2ad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:25.984658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebafddd-7533-441f-bf78-2778acc2ad67 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:25.984938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebafddd-7533-441f-bf78-2778acc2ad67 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:25.985212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebafddd-7533-441f-bf78-2778acc2ad67 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:25.985844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebafddd-7533-441f-bf78-2778acc2ad67 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:25.985933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e5781-3835-40bc-8cc4-b66cee0e18cf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:25.987290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebafddd-7533-441f-bf78-2778acc2ad67 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:26.015082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 25/54] 10.222.0.86 () {60 vars in 1140 bytes} [Wed Aug 24 09:49:25 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:49:26.022761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07b425-c696-494c-8f29-9d8b152b4927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:26.023415 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07b425-c696-494c-8f29-9d8b152b4927 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:26.023699 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07b425-c696-494c-8f29-9d8b152b4927 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:26.023981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07b425-c696-494c-8f29-9d8b152b4927 None admin] PATH_INFO: `/v3/users/9757335c28f74ea5a8fbdedc0ae7682a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:26.025241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07b425-c696-494c-8f29-9d8b152b4927 None admin] RBAC: Authorizing `identity:get_user(user_id=9757335c28f74ea5a8fbdedc0ae7682a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:26.026757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07b425-c696-494c-8f29-9d8b152b4927 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:26.028497 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 30/55] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:25 2022] GET /identity/v3/users/9757335c28f74ea5a8fbdedc0ae7682a => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:26.034651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebafddd-7533-441f-bf78-2778acc2ad67 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:26.147745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebafddd-7533-441f-bf78-2778acc2ad67 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:26.165701 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:26.184786 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a7e6d-b542-47d0-9e97-812d56bcc4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:26.185910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a7e6d-b542-47d0-9e97-812d56bcc4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:26.186191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a7e6d-b542-47d0-9e97-812d56bcc4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:26.186472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a7e6d-b542-47d0-9e97-812d56bcc4e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:26.187565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a7e6d-b542-47d0-9e97-812d56bcc4e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:26.189034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a7e6d-b542-47d0-9e97-812d56bcc4e7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:26.190599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 26/56] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:26.196629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07b425-c696-494c-8f29-9d8b152b4927 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:26.276536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:26.302072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07b425-c696-494c-8f29-9d8b152b4927 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:26.337369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd264b-1fca-4ea5-b8b2-354478c92c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:26.337974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd264b-1fca-4ea5-b8b2-354478c92c41 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:26.338258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd264b-1fca-4ea5-b8b2-354478c92c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:26.338534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd264b-1fca-4ea5-b8b2-354478c92c41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:26.339263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd264b-1fca-4ea5-b8b2-354478c92c41 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.domain.id=default, user.id=9757335c28f74ea5a8fbdedc0ae7682a) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:26.339576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd264b-1fca-4ea5-b8b2-354478c92c41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:26.341067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd264b-1fca-4ea5-b8b2-354478c92c41 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:26.346844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 31/57] 10.222.0.86 () {56 vars in 1322 bytes} [Wed Aug 24 09:49:26 2022] GET /identity/v3/role_assignments?user.id=9757335c28f74ea5a8fbdedc0ae7682a&scope.domain.id=default&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:27.931296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59603da9-a1db-4cc9-ac5a-36c869ced5e8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:27.931806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59603da9-a1db-4cc9-ac5a-36c869ced5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:27.931919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59603da9-a1db-4cc9-ac5a-36c869ced5e8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:27.933221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 27/58] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:27.938925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12f827-132a-4954-a7ea-e5a9787aa476 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:27.939295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12f827-132a-4954-a7ea-e5a9787aa476 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:27.939475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12f827-132a-4954-a7ea-e5a9787aa476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:27.957908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ae12f827-132a-4954-a7ea-e5a9787aa476 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:27.959102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ae12f827-132a-4954-a7ea-e5a9787aa476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:28.023063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 32/59] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:28.292944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59603da9-a1db-4cc9-ac5a-36c869ced5e8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:28.392608 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59603da9-a1db-4cc9-ac5a-36c869ced5e8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:28.424932 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff0078-f382-4cad-a052-2fc05851edea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:28.425548 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff0078-f382-4cad-a052-2fc05851edea None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:28.425835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff0078-f382-4cad-a052-2fc05851edea None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:28.426116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff0078-f382-4cad-a052-2fc05851edea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:28.427228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff0078-f382-4cad-a052-2fc05851edea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:28.428710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff0078-f382-4cad-a052-2fc05851edea None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:28.430281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 28/60] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:28.436244 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12f827-132a-4954-a7ea-e5a9787aa476 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:28.502390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:28.533026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12f827-132a-4954-a7ea-e5a9787aa476 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:28.566214 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310f2cc-ae2d-4485-b8c4-90a830371e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:28.566749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310f2cc-ae2d-4485-b8c4-90a830371e13 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:28.567032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310f2cc-ae2d-4485-b8c4-90a830371e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:28.567306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310f2cc-ae2d-4485-b8c4-90a830371e13 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:28.567939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310f2cc-ae2d-4485-b8c4-90a830371e13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:28.569433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310f2cc-ae2d-4485-b8c4-90a830371e13 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:28.596059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 33/61] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:49:28 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:49:31.479928 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6548-fe5f-45a8-a4f4-12f57053083e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:31.480499 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6548-fe5f-45a8-a4f4-12f57053083e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:31.480762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6548-fe5f-45a8-a4f4-12f57053083e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:31.483415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 29/62] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:31.488710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f50a-d640-4c80-8393-6fe37f35024d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:31.489555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f50a-d640-4c80-8393-6fe37f35024d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:31.490030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f50a-d640-4c80-8393-6fe37f35024d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:31.517020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-9f70f50a-d640-4c80-8393-6fe37f35024d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:31.518564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9f70f50a-d640-4c80-8393-6fe37f35024d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:31.611159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 34/63] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:31.720582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861c3b2-f547-4b34-82a5-00a3cce9a4be None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:31.720863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861c3b2-f547-4b34-82a5-00a3cce9a4be None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:31.721321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861c3b2-f547-4b34-82a5-00a3cce9a4be None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:31.723239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 30/64] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:31.728358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53ffed-fef3-49e4-b0b8-9cb096287807 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:31.728653 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53ffed-fef3-49e4-b0b8-9cb096287807 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:31.728937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53ffed-fef3-49e4-b0b8-9cb096287807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:31.757132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ef53ffed-fef3-49e4-b0b8-9cb096287807 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:31.758415 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ef53ffed-fef3-49e4-b0b8-9cb096287807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:31.831009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:31.859005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 35/65] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:31.973269 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f27410-e2fe-424e-9b26-32f9c088657b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:31.973952 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f27410-e2fe-424e-9b26-32f9c088657b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:31.974422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f27410-e2fe-424e-9b26-32f9c088657b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:31.975979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 31/66] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:31.981619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d0d05-b348-4eff-84fd-ff88a3151e0a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:31.981901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d0d05-b348-4eff-84fd-ff88a3151e0a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:31.982183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d0d05-b348-4eff-84fd-ff88a3151e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.009170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2e3d0d05-b348-4eff-84fd-ff88a3151e0a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:32.010300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2e3d0d05-b348-4eff-84fd-ff88a3151e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:32.022365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f27410-e2fe-424e-9b26-32f9c088657b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:32.124809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 36/67] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:32.125182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:32.187009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f27410-e2fe-424e-9b26-32f9c088657b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:32.223301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e26ed8-ad1a-47cf-ba19-5896cdedb099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:32.224062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e26ed8-ad1a-47cf-ba19-5896cdedb099 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.224407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e26ed8-ad1a-47cf-ba19-5896cdedb099 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.224692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e26ed8-ad1a-47cf-ba19-5896cdedb099 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.233954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f697a-c13e-4663-9349-a7c0ef83a60a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.234249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f697a-c13e-4663-9349-a7c0ef83a60a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.234529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f697a-c13e-4663-9349-a7c0ef83a60a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.236168 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 37/68] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:32.241589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e0c7a-0b1b-4b43-bb0e-8188a88f4b92 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.241882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e0c7a-0b1b-4b43-bb0e-8188a88f4b92 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.242169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e0c7a-0b1b-4b43-bb0e-8188a88f4b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.244865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e26ed8-ad1a-47cf-ba19-5896cdedb099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b0e26ed8-ad1a-47cf-ba19-5896cdedb099 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:32.256099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:32.262140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:32.263993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 32/69] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:49:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:49:32.266231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e26ed8-ad1a-47cf-ba19-5896cdedb099 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:32.272514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-574e0c7a-0b1b-4b43-bb0e-8188a88f4b92 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:32.274650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-574e0c7a-0b1b-4b43-bb0e-8188a88f4b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:32.367858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 38/70] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:32.368760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e0c7a-0b1b-4b43-bb0e-8188a88f4b92 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:32.383194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e26ed8-ad1a-47cf-ba19-5896cdedb099 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:32.397515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:32.420578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c25888-579d-4b87-9387-505647e1b733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:32.421783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c25888-579d-4b87-9387-505647e1b733 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.422074 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c25888-579d-4b87-9387-505647e1b733 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.422353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c25888-579d-4b87-9387-505647e1b733 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.423031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c25888-579d-4b87-9387-505647e1b733 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:32.423341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c25888-579d-4b87-9387-505647e1b733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:32.424778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c25888-579d-4b87-9387-505647e1b733 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:32.427791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:32.439745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 33/71] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:32.439745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42423f78-853f-4669-ae7e-e1d1aeae3913 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.439745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42423f78-853f-4669-ae7e-e1d1aeae3913 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.439745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42423f78-853f-4669-ae7e-e1d1aeae3913 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.439745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 34/72] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:32.439745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42423f78-853f-4669-ae7e-e1d1aeae3913 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:32.488343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e0c7a-0b1b-4b43-bb0e-8188a88f4b92 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:32.525015 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c4345-b3b0-4747-9869-deae5c049809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:32.526374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c4345-b3b0-4747-9869-deae5c049809 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.526738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c4345-b3b0-4747-9869-deae5c049809 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.527014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c4345-b3b0-4747-9869-deae5c049809 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.537383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c4345-b3b0-4747-9869-deae5c049809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ad8c4345-b3b0-4747-9869-deae5c049809 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:32.542248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:32.543085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:32.543847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 39/73] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:32.545310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c4345-b3b0-4747-9869-deae5c049809 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:32.561963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42423f78-853f-4669-ae7e-e1d1aeae3913 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:32.604653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a436f9-1d63-48e5-92bc-ff401cd39157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:32.605286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a436f9-1d63-48e5-92bc-ff401cd39157 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.605567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a436f9-1d63-48e5-92bc-ff401cd39157 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.605847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a436f9-1d63-48e5-92bc-ff401cd39157 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.617177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a436f9-1d63-48e5-92bc-ff401cd39157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-74a436f9-1d63-48e5-92bc-ff401cd39157 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:32.621944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:32.622939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:32.623754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 35/74] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:49:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:32.625012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84431-1d2f-4927-b16a-86363d4b62b7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.625228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84431-1d2f-4927-b16a-86363d4b62b7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.625506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84431-1d2f-4927-b16a-86363d4b62b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.659797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c4345-b3b0-4747-9869-deae5c049809 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:32.668939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:32.698859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c191fd-900c-4baa-9b8b-8aa6d31d7fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:32.699499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c191fd-900c-4baa-9b8b-8aa6d31d7fde None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.699781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c191fd-900c-4baa-9b8b-8aa6d31d7fde None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.700077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c191fd-900c-4baa-9b8b-8aa6d31d7fde None admin] PATH_INFO: `/v3/users/9757335c28f74ea5a8fbdedc0ae7682a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.701245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c191fd-900c-4baa-9b8b-8aa6d31d7fde None admin] RBAC: Authorizing `identity:get_user(user_id=9757335c28f74ea5a8fbdedc0ae7682a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:32.702785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c191fd-900c-4baa-9b8b-8aa6d31d7fde None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:32.704499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 40/75] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:32 2022] GET /identity/v3/users/9757335c28f74ea5a8fbdedc0ae7682a => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:32.705291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c191fd-900c-4baa-9b8b-8aa6d31d7fde None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:32.748391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f7f84431-1d2f-4927-b16a-86363d4b62b7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:32.749609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f7f84431-1d2f-4927-b16a-86363d4b62b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:32.775661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:32.811097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c191fd-900c-4baa-9b8b-8aa6d31d7fde None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:32.817665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 36/76] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:32.818506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84431-1d2f-4927-b16a-86363d4b62b7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:32.849041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d98fd7-0318-4a93-aa51-a52b462ec26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:32.849650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d98fd7-0318-4a93-aa51-a52b462ec26c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.849936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d98fd7-0318-4a93-aa51-a52b462ec26c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.850219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d98fd7-0318-4a93-aa51-a52b462ec26c None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.850876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d98fd7-0318-4a93-aa51-a52b462ec26c None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:32.851183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d98fd7-0318-4a93-aa51-a52b462ec26c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:32.852616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d98fd7-0318-4a93-aa51-a52b462ec26c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:32.861947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 41/77] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:32.864921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d98fd7-0318-4a93-aa51-a52b462ec26c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:32.902465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:32.928187 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84431-1d2f-4927-b16a-86363d4b62b7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:32.963288 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f46f1-aa1a-4624-b0f9-a0ce61f75edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:32.963896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f46f1-aa1a-4624-b0f9-a0ce61f75edd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.964197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f46f1-aa1a-4624-b0f9-a0ce61f75edd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.964468 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f46f1-aa1a-4624-b0f9-a0ce61f75edd None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.965236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f46f1-aa1a-4624-b0f9-a0ce61f75edd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:32.965561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f46f1-aa1a-4624-b0f9-a0ce61f75edd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:32.967004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f46f1-aa1a-4624-b0f9-a0ce61f75edd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:32.972011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d98fd7-0318-4a93-aa51-a52b462ec26c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:32.976195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 37/78] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:49:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:32.977318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f46f1-aa1a-4624-b0f9-a0ce61f75edd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.007555 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:33.011240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb28ba6-6ec5-4a6e-9689-538f29aa5027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:33.011818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb28ba6-6ec5-4a6e-9689-538f29aa5027 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:33.012124 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb28ba6-6ec5-4a6e-9689-538f29aa5027 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:33.012658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb28ba6-6ec5-4a6e-9689-538f29aa5027 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:33.023159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb28ba6-6ec5-4a6e-9689-538f29aa5027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-fdb28ba6-6ec5-4a6e-9689-538f29aa5027 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:33.028348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:33.029207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:33.030036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 42/79] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:33.031197 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb28ba6-6ec5-4a6e-9689-538f29aa5027 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.089989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f46f1-aa1a-4624-b0f9-a0ce61f75edd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:33.123464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44c96a-8725-4545-afeb-80f84b25a8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:33.124128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44c96a-8725-4545-afeb-80f84b25a8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:33.124415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44c96a-8725-4545-afeb-80f84b25a8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:33.124702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44c96a-8725-4545-afeb-80f84b25a8f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:33.128030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:33.137470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb28ba6-6ec5-4a6e-9689-538f29aa5027 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:33.171726 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a5f8c-388d-4086-8a25-73fbfbcac8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:33.172387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a5f8c-388d-4086-8a25-73fbfbcac8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:33.172674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a5f8c-388d-4086-8a25-73fbfbcac8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:33.172953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a5f8c-388d-4086-8a25-73fbfbcac8f4 None admin] PATH_INFO: `/v3/users/7d7bdddb7df24e8a9615b6909130f4fc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:33.185991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a5f8c-388d-4086-8a25-73fbfbcac8f4 None admin] RBAC: Authorizing `identity:get_user(user_id=7d7bdddb7df24e8a9615b6909130f4fc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:33.187478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a5f8c-388d-4086-8a25-73fbfbcac8f4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:33.189147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 43/80] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:33 2022] GET /identity/v3/users/7d7bdddb7df24e8a9615b6909130f4fc => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:33.189924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a5f8c-388d-4086-8a25-73fbfbcac8f4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.191898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44c96a-8725-4545-afeb-80f84b25a8f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:33.193534 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44c96a-8725-4545-afeb-80f84b25a8f7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:33.220116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:33.287921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a5f8c-388d-4086-8a25-73fbfbcac8f4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:33.293064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 38/81] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:33.293905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44c96a-8725-4545-afeb-80f84b25a8f7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.324195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e0ed3-0618-452c-a770-e43c1d85aca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:33.324823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e0ed3-0618-452c-a770-e43c1d85aca8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:33.325105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e0ed3-0618-452c-a770-e43c1d85aca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:33.325379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e0ed3-0618-452c-a770-e43c1d85aca8 None admin] PATH_INFO: `/v3/users/7d7bdddb7df24e8a9615b6909130f4fc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:33.326524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e0ed3-0618-452c-a770-e43c1d85aca8 None admin] RBAC: Authorizing `identity:get_user(user_id=7d7bdddb7df24e8a9615b6909130f4fc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:33.328491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e0ed3-0618-452c-a770-e43c1d85aca8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:33.330070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 44/82] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:33 2022] GET /identity/v3/users/7d7bdddb7df24e8a9615b6909130f4fc => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:33.330825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e0ed3-0618-452c-a770-e43c1d85aca8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.354145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:33.398658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44c96a-8725-4545-afeb-80f84b25a8f7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:33.424699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:33.430371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e0ed3-0618-452c-a770-e43c1d85aca8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:33.433458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a7b75-5e2d-43a8-ae26-63bf52388723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:33.434073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a7b75-5e2d-43a8-ae26-63bf52388723 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:33.434360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a7b75-5e2d-43a8-ae26-63bf52388723 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:33.434639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a7b75-5e2d-43a8-ae26-63bf52388723 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:33.435311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a7b75-5e2d-43a8-ae26-63bf52388723 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:33.435616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a7b75-5e2d-43a8-ae26-63bf52388723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:33.437562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a7b75-5e2d-43a8-ae26-63bf52388723 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:33.445938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 39/83] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:33.450559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a7b75-5e2d-43a8-ae26-63bf52388723 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.468379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6ddf1-725a-444f-a1ea-5602069277ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:33.469007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6ddf1-725a-444f-a1ea-5602069277ab None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:33.469286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6ddf1-725a-444f-a1ea-5602069277ab None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:33.469560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6ddf1-725a-444f-a1ea-5602069277ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:33.470657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6ddf1-725a-444f-a1ea-5602069277ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:33.472165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6ddf1-725a-444f-a1ea-5602069277ab None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:33.473725 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 45/84] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:33.474473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6ddf1-725a-444f-a1ea-5602069277ab None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.548963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a7b75-5e2d-43a8-ae26-63bf52388723 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:33.564223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:33.573932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6ddf1-725a-444f-a1ea-5602069277ab None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:33.582917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d4d9a-9bc3-4c11-a977-76bdd3efb3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:33.583527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4d9a-9bc3-4c11-a977-76bdd3efb3fc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:33.583811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4d9a-9bc3-4c11-a977-76bdd3efb3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:33.584114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4d9a-9bc3-4c11-a977-76bdd3efb3fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:33.608314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896632d6-c9f6-44a0-9311-e7f7e0860de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:33.609134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896632d6-c9f6-44a0-9311-e7f7e0860de0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:33.609426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896632d6-c9f6-44a0-9311-e7f7e0860de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:33.609729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896632d6-c9f6-44a0-9311-e7f7e0860de0 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:33.610984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896632d6-c9f6-44a0-9311-e7f7e0860de0 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:33.612496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896632d6-c9f6-44a0-9311-e7f7e0860de0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:33.614151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 46/85] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:33 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:33.614915 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896632d6-c9f6-44a0-9311-e7f7e0860de0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.640743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:33.653892 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d4d9a-9bc3-4c11-a977-76bdd3efb3fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:33.655367 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d4d9a-9bc3-4c11-a977-76bdd3efb3fc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:33.715950 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896632d6-c9f6-44a0-9311-e7f7e0860de0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:33.749854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f369b-eaba-4469-aa68-736d05ecc50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:33.750456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f369b-eaba-4469-aa68-736d05ecc50c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:33.750740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f369b-eaba-4469-aa68-736d05ecc50c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:33.751018 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f369b-eaba-4469-aa68-736d05ecc50c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:33.753818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 40/86] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:33.754651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d4d9a-9bc3-4c11-a977-76bdd3efb3fc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.779573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:33.823899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f369b-eaba-4469-aa68-736d05ecc50c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:33.825347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f369b-eaba-4469-aa68-736d05ecc50c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:33.844163 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:33.857056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d4d9a-9bc3-4c11-a977-76bdd3efb3fc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:33.891457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657ed27-e8c8-4921-933d-57f5ee415406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:33.892114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657ed27-e8c8-4921-933d-57f5ee415406 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:33.892401 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657ed27-e8c8-4921-933d-57f5ee415406 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:33.892667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657ed27-e8c8-4921-933d-57f5ee415406 None admin] PATH_INFO: `/v3/users/7d7bdddb7df24e8a9615b6909130f4fc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:33.893778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657ed27-e8c8-4921-933d-57f5ee415406 None admin] RBAC: Authorizing `identity:get_user(user_id=7d7bdddb7df24e8a9615b6909130f4fc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:33.895208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657ed27-e8c8-4921-933d-57f5ee415406 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:33.896811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 41/87] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:33 2022] GET /identity/v3/users/7d7bdddb7df24e8a9615b6909130f4fc => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:33.897553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657ed27-e8c8-4921-933d-57f5ee415406 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.925399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 47/88] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:33.925399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f369b-eaba-4469-aa68-736d05ecc50c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.987039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:33.998007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657ed27-e8c8-4921-933d-57f5ee415406 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:34.027769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f369b-eaba-4469-aa68-736d05ecc50c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:34.031058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab437888-fa1a-4239-8d9b-76daac23e6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:34.031660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437888-fa1a-4239-8d9b-76daac23e6d8 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:34.031939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437888-fa1a-4239-8d9b-76daac23e6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:34.032243 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437888-fa1a-4239-8d9b-76daac23e6d8 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:34.032879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab437888-fa1a-4239-8d9b-76daac23e6d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:34.034316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab437888-fa1a-4239-8d9b-76daac23e6d8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:34.062748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 42/89] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:49:33 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:49:34.062911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b7f74-4620-4eb5-87c6-bd8f057adb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:34.063397 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b7f74-4620-4eb5-87c6-bd8f057adb75 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:34.063560 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab437888-fa1a-4239-8d9b-76daac23e6d8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:34.063707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b7f74-4620-4eb5-87c6-bd8f057adb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:34.064034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b7f74-4620-4eb5-87c6-bd8f057adb75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:34.064803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b7f74-4620-4eb5-87c6-bd8f057adb75 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=2ec79012680c43e3bfc414986fbb0b79, user.id=9757335c28f74ea5a8fbdedc0ae7682a) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:34.065118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b7f74-4620-4eb5-87c6-bd8f057adb75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:34.066617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b7f74-4620-4eb5-87c6-bd8f057adb75 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:34.072740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 48/90] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:33 2022] GET /identity/v3/role_assignments?user.id=9757335c28f74ea5a8fbdedc0ae7682a&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:34.073620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b7f74-4620-4eb5-87c6-bd8f057adb75 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:34.112604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:34.208667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b7f74-4620-4eb5-87c6-bd8f057adb75 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:34.236417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:34.244204 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab437888-fa1a-4239-8d9b-76daac23e6d8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:34.252018 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15ac5e-a6a6-4975-a3f5-2b7af9a94a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:34.252670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15ac5e-a6a6-4975-a3f5-2b7af9a94a69 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:34.252948 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15ac5e-a6a6-4975-a3f5-2b7af9a94a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:34.253217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15ac5e-a6a6-4975-a3f5-2b7af9a94a69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:34.292255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2a3e3-673b-4341-9c91-f9431ce32832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:34.293134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2a3e3-673b-4341-9c91-f9431ce32832 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:34.293559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2a3e3-673b-4341-9c91-f9431ce32832 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:34.294020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2a3e3-673b-4341-9c91-f9431ce32832 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:34.295181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2a3e3-673b-4341-9c91-f9431ce32832 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:34.297492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2a3e3-673b-4341-9c91-f9431ce32832 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:34.299087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 43/91] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:34 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:34.303076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2a3e3-673b-4341-9c91-f9431ce32832 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:34.337268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15ac5e-a6a6-4975-a3f5-2b7af9a94a69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:34.338779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15ac5e-a6a6-4975-a3f5-2b7af9a94a69 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:34.391909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:34.419782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2a3e3-673b-4341-9c91-f9431ce32832 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:34.455524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 49/92] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:34.457506 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15ac5e-a6a6-4975-a3f5-2b7af9a94a69 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:34.471028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161007d9-ddb4-48a2-bc66-a903853ab23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:34.471673 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161007d9-ddb4-48a2-bc66-a903853ab23b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:34.471950 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161007d9-ddb4-48a2-bc66-a903853ab23b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:34.472313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161007d9-ddb4-48a2-bc66-a903853ab23b None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:34.474025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161007d9-ddb4-48a2-bc66-a903853ab23b None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:34.475470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161007d9-ddb4-48a2-bc66-a903853ab23b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:34.477229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 44/93] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:34 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:34.478015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161007d9-ddb4-48a2-bc66-a903853ab23b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:34.496634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:34.600286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15ac5e-a6a6-4975-a3f5-2b7af9a94a69 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:34.618822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161007d9-ddb4-48a2-bc66-a903853ab23b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:34.638931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f4f04-5a8d-4ab9-bf5e-546f9dd9ba52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:34.639985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f4f04-5a8d-4ab9-bf5e-546f9dd9ba52 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:34.640702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f4f04-5a8d-4ab9-bf5e-546f9dd9ba52 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:34.641201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f4f04-5a8d-4ab9-bf5e-546f9dd9ba52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:34.642271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f4f04-5a8d-4ab9-bf5e-546f9dd9ba52 None admin] RBAC: Adding query filter params (role.id=7ab2687826544b8e80c709d6061162d4, scope.project.id=2ec79012680c43e3bfc414986fbb0b79, user.id=7d7bdddb7df24e8a9615b6909130f4fc) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:34.642781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f4f04-5a8d-4ab9-bf5e-546f9dd9ba52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:34.644484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f4f04-5a8d-4ab9-bf5e-546f9dd9ba52 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:34.650580 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 50/94] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:34 2022] GET /identity/v3/role_assignments?user.id=7d7bdddb7df24e8a9615b6909130f4fc&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=7ab2687826544b8e80c709d6061162d4 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:34.651630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f4f04-5a8d-4ab9-bf5e-546f9dd9ba52 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:34.658206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccbcd0-619a-40c4-b12e-f4e81f8d3eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:34.658873 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccbcd0-619a-40c4-b12e-f4e81f8d3eec None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:34.659158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccbcd0-619a-40c4-b12e-f4e81f8d3eec None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:34.659436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccbcd0-619a-40c4-b12e-f4e81f8d3eec None admin] PATH_INFO: `/v3/projects/44fd0577657f4aa0838705dc26102320` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:34.660723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccbcd0-619a-40c4-b12e-f4e81f8d3eec None admin] RBAC: Authorizing `identity:get_project(project_id=44fd0577657f4aa0838705dc26102320)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:34.662550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccbcd0-619a-40c4-b12e-f4e81f8d3eec None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:34.664767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 45/95] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:34 2022] GET /identity/v3/projects/44fd0577657f4aa0838705dc26102320 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:34.677371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccbcd0-619a-40c4-b12e-f4e81f8d3eec None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:34.708775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:34.753264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:34.770517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f4f04-5a8d-4ab9-bf5e-546f9dd9ba52 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:34.816657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee216bd8-5604-4a2c-8eff-876cdb810317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:34.817521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee216bd8-5604-4a2c-8eff-876cdb810317 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:34.817946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee216bd8-5604-4a2c-8eff-876cdb810317 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:34.818532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee216bd8-5604-4a2c-8eff-876cdb810317 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:34.819442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee216bd8-5604-4a2c-8eff-876cdb810317 None admin] RBAC: Adding query filter params (role.id=fc1186ecd0f74b2fad8155d528f268f9, scope.project.id=2ec79012680c43e3bfc414986fbb0b79, user.id=7d7bdddb7df24e8a9615b6909130f4fc) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:34.820897 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee216bd8-5604-4a2c-8eff-876cdb810317 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:34.822540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee216bd8-5604-4a2c-8eff-876cdb810317 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:34.829669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 51/96] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:34 2022] GET /identity/v3/role_assignments?user.id=7d7bdddb7df24e8a9615b6909130f4fc&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=fc1186ecd0f74b2fad8155d528f268f9 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:34.885317 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccbcd0-619a-40c4-b12e-f4e81f8d3eec None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:34.946075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920a075-2680-4f7e-8e28-96a3f2e456b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:34.946755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920a075-2680-4f7e-8e28-96a3f2e456b7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:34.947037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920a075-2680-4f7e-8e28-96a3f2e456b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:34.947307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920a075-2680-4f7e-8e28-96a3f2e456b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:34.948065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920a075-2680-4f7e-8e28-96a3f2e456b7 None admin] RBAC: Adding query filter params (role.id=7ab2687826544b8e80c709d6061162d4, scope.project.id=44fd0577657f4aa0838705dc26102320, user.id=7d7bdddb7df24e8a9615b6909130f4fc) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:34.948380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920a075-2680-4f7e-8e28-96a3f2e456b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:34.949855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920a075-2680-4f7e-8e28-96a3f2e456b7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:34.959815 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 46/97] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:34 2022] GET /identity/v3/role_assignments?user.id=7d7bdddb7df24e8a9615b6909130f4fc&scope.project.id=44fd0577657f4aa0838705dc26102320&role.id=7ab2687826544b8e80c709d6061162d4 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:36.571195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca091623-8a75-4f3a-a84d-58d4c78c4776 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:36.571706 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca091623-8a75-4f3a-a84d-58d4c78c4776 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:36.571780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca091623-8a75-4f3a-a84d-58d4c78c4776 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:36.573138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 52/98] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:36.579064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42ab80-d7ab-4bc6-91e8-52be4d4985c2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:36.579484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42ab80-d7ab-4bc6-91e8-52be4d4985c2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:36.579684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42ab80-d7ab-4bc6-91e8-52be4d4985c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:36.608223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ed42ab80-d7ab-4bc6-91e8-52be4d4985c2 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:36.609787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ed42ab80-d7ab-4bc6-91e8-52be4d4985c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:36.695188 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 47/99] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:37.013404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f05f8-51dc-4afb-a461-cfd59505d176 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:37.014108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f05f8-51dc-4afb-a461-cfd59505d176 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:37.014595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f05f8-51dc-4afb-a461-cfd59505d176 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:37.016292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 53/100] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:37.021733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2368e83-724f-444e-b206-259cb9a58a3c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:37.022279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2368e83-724f-444e-b206-259cb9a58a3c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:37.022767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2368e83-724f-444e-b206-259cb9a58a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:37.041407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b2368e83-724f-444e-b206-259cb9a58a3c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:37.042896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b2368e83-724f-444e-b206-259cb9a58a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:37.046052 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:37.089508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196514-b22f-4900-9db8-bf474e5acfa9 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:37.089817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196514-b22f-4900-9db8-bf474e5acfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:37.090073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196514-b22f-4900-9db8-bf474e5acfa9 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:37.091597 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 54/101] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:49:37.099882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1c83f-c624-40af-9953-e49587026fb6 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:37.099882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1c83f-c624-40af-9953-e49587026fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:37.099882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1c83f-c624-40af-9953-e49587026fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:37.119466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-afa1c83f-c624-40af-9953-e49587026fb6 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:37.121040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-afa1c83f-c624-40af-9953-e49587026fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:37.181398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 48/102] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:37.196389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 55/103] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:37.242969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2368e83-724f-444e-b206-259cb9a58a3c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:37.384130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2368e83-724f-444e-b206-259cb9a58a3c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:37.419526 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4401f-8b59-40ea-b6b5-149063170a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:37.420196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4401f-8b59-40ea-b6b5-149063170a04 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:37.420483 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4401f-8b59-40ea-b6b5-149063170a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:37.420759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4401f-8b59-40ea-b6b5-149063170a04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:37.436214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4401f-8b59-40ea-b6b5-149063170a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:37.440486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c1e4401f-8b59-40ea-b6b5-149063170a04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:37.446056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:37.447016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:37.447787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:37.448403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 49/104] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:49:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:37.453862 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1c83f-c624-40af-9953-e49587026fb6 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:37.552899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4401f-8b59-40ea-b6b5-149063170a04 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:37.592408 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:37.666238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4401f-8b59-40ea-b6b5-149063170a04 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:37.677968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1c83f-c624-40af-9953-e49587026fb6 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:37.703914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a2170-9484-489b-bd6f-9923bd6d0d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:37.704593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a2170-9484-489b-bd6f-9923bd6d0d31 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:37.704878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a2170-9484-489b-bd6f-9923bd6d0d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:37.705156 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a2170-9484-489b-bd6f-9923bd6d0d31 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:37.716859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a2170-9484-489b-bd6f-9923bd6d0d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-f98a2170-9484-489b-bd6f-9923bd6d0d31 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:37.722758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:37.723617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:37.724446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 50/105] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:49:37.729363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a2170-9484-489b-bd6f-9923bd6d0d31 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:37.798584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:37.801733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aba62e-4d8b-4750-a830-4bbce4d753e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:37.802560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aba62e-4d8b-4750-a830-4bbce4d753e3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:37.802847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aba62e-4d8b-4750-a830-4bbce4d753e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:37.803123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aba62e-4d8b-4750-a830-4bbce4d753e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:37.803843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aba62e-4d8b-4750-a830-4bbce4d753e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:37.804178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aba62e-4d8b-4750-a830-4bbce4d753e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:37.805592 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aba62e-4d8b-4750-a830-4bbce4d753e3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:37.823895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 56/106] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:37.824785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aba62e-4d8b-4750-a830-4bbce4d753e3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:37.850797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a2170-9484-489b-bd6f-9923bd6d0d31 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:37.888491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1112e-d23a-458b-96c1-991ae3a90edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:37.889378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1112e-d23a-458b-96c1-991ae3a90edc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:37.889668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1112e-d23a-458b-96c1-991ae3a90edc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:37.890814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1112e-d23a-458b-96c1-991ae3a90edc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:37.892269 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1112e-d23a-458b-96c1-991ae3a90edc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:37.893739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1112e-d23a-458b-96c1-991ae3a90edc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:37.900148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 51/107] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:37.901756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45a63f-3d4d-452e-81b9-af298160aff6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:37.902043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45a63f-3d4d-452e-81b9-af298160aff6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:37.902321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45a63f-3d4d-452e-81b9-af298160aff6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:37.903753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 52/108] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:37.905134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45a63f-3d4d-452e-81b9-af298160aff6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:37.941836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aba62e-4d8b-4750-a830-4bbce4d753e3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:37.981352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9c02a-4f24-4de6-bcef-44e9297a40c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:37.981966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9c02a-4f24-4de6-bcef-44e9297a40c4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:37.982252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9c02a-4f24-4de6-bcef-44e9297a40c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:37.982531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9c02a-4f24-4de6-bcef-44e9297a40c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:37.983341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9c02a-4f24-4de6-bcef-44e9297a40c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:37.983666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9c02a-4f24-4de6-bcef-44e9297a40c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:37.986043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9c02a-4f24-4de6-bcef-44e9297a40c4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:37.995634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 57/109] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:37.996455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9c02a-4f24-4de6-bcef-44e9297a40c4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:38.004294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:38.020720 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45a63f-3d4d-452e-81b9-af298160aff6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:38.046949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:38.055127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146b8e0-eb08-4616-8867-868f6b0e62cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:38.055770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146b8e0-eb08-4616-8867-868f6b0e62cc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.056063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146b8e0-eb08-4616-8867-868f6b0e62cc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.056355 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146b8e0-eb08-4616-8867-868f6b0e62cc None admin] PATH_INFO: `/v3/users/9757335c28f74ea5a8fbdedc0ae7682a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:38.057547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146b8e0-eb08-4616-8867-868f6b0e62cc None admin] RBAC: Authorizing `identity:get_user(user_id=9757335c28f74ea5a8fbdedc0ae7682a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:38.058996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146b8e0-eb08-4616-8867-868f6b0e62cc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:38.060636 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 53/110] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:37 2022] GET /identity/v3/users/9757335c28f74ea5a8fbdedc0ae7682a => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:38.065377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e26ab-985d-4f84-b34d-819bf65c1e13 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.065806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e26ab-985d-4f84-b34d-819bf65c1e13 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.066087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e26ab-985d-4f84-b34d-819bf65c1e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:38.084402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e76e26ab-985d-4f84-b34d-819bf65c1e13 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:38.085816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e76e26ab-985d-4f84-b34d-819bf65c1e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:38.104687 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9c02a-4f24-4de6-bcef-44e9297a40c4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:38.137393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872c008-bace-478d-a426-a9d87a2e48df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:38.137989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872c008-bace-478d-a426-a9d87a2e48df None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.138269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872c008-bace-478d-a426-a9d87a2e48df None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.138540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872c008-bace-478d-a426-a9d87a2e48df None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:38.139520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872c008-bace-478d-a426-a9d87a2e48df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:38.141268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872c008-bace-478d-a426-a9d87a2e48df None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:38.153264 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 54/111] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:38.154514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49601f-4ee6-4edd-9acb-2705d5045370 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.154807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49601f-4ee6-4edd-9acb-2705d5045370 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.155086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49601f-4ee6-4edd-9acb-2705d5045370 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:38.156575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 55/112] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:38.161728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e7c7b-0a64-4d32-b7f0-723d13faf8c0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.162013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e7c7b-0a64-4d32-b7f0-723d13faf8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.162548 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e7c7b-0a64-4d32-b7f0-723d13faf8c0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:38.163792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 56/113] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:38.164684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e7c7b-0a64-4d32-b7f0-723d13faf8c0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:38.173039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 58/114] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:49:37 2022] POST /identity/v3/projects => generated 302 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:49:38.174564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872c008-bace-478d-a426-a9d87a2e48df None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:38.271327 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:38.301553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e7c7b-0a64-4d32-b7f0-723d13faf8c0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:38.337607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872c008-bace-478d-a426-a9d87a2e48df None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:38.343738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:38.355834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64a757-0c50-4622-a52e-76cc588199de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:38.357040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64a757-0c50-4622-a52e-76cc588199de None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.357328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64a757-0c50-4622-a52e-76cc588199de None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.357613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64a757-0c50-4622-a52e-76cc588199de None admin] PATH_INFO: `/v3/users/d70bcc9f75eb448e807921f69100128f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:38.378257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64a757-0c50-4622-a52e-76cc588199de None admin] RBAC: Authorizing `identity:get_user(user_id=d70bcc9f75eb448e807921f69100128f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:38.380159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64a757-0c50-4622-a52e-76cc588199de None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:38.382059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 57/115] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:38 2022] GET /identity/v3/users/d70bcc9f75eb448e807921f69100128f => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:38.383579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35b7ff-40d3-41d2-a1c7-286301008402 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.383940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35b7ff-40d3-41d2-a1c7-286301008402 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.384322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35b7ff-40d3-41d2-a1c7-286301008402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:38.388885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac1896-a3b5-466c-a364-86d1997780bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:38.389760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac1896-a3b5-466c-a364-86d1997780bc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.390241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac1896-a3b5-466c-a364-86d1997780bc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.390907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac1896-a3b5-466c-a364-86d1997780bc None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:38.392921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac1896-a3b5-466c-a364-86d1997780bc None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:38.394689 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac1896-a3b5-466c-a364-86d1997780bc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:38.396771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 59/116] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:38 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:38.398230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785cd73-a7c4-43b9-bb49-2942dfd209f4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.398522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785cd73-a7c4-43b9-bb49-2942dfd209f4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.398812 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785cd73-a7c4-43b9-bb49-2942dfd209f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:38.407115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0f35b7ff-40d3-41d2-a1c7-286301008402 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:38.408897 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0f35b7ff-40d3-41d2-a1c7-286301008402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:38.418178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4785cd73-a7c4-43b9-bb49-2942dfd209f4 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:38.418831 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4785cd73-a7c4-43b9-bb49-2942dfd209f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:38.490683 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 60/117] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:38.491082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 58/118] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:38.491909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785cd73-a7c4-43b9-bb49-2942dfd209f4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:38.492761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35b7ff-40d3-41d2-a1c7-286301008402 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:38.587412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:38.689106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785cd73-a7c4-43b9-bb49-2942dfd209f4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:38.703951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35b7ff-40d3-41d2-a1c7-286301008402 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:38.733819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:38.755998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780d53f-8789-4cdf-abbe-3469031ac762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:38.756641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780d53f-8789-4cdf-abbe-3469031ac762 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.756924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780d53f-8789-4cdf-abbe-3469031ac762 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.757200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780d53f-8789-4cdf-abbe-3469031ac762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:38.788001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b317e-12bf-4243-8101-e337c334a88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:38.788826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b317e-12bf-4243-8101-e337c334a88d None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.789117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b317e-12bf-4243-8101-e337c334a88d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.789403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b317e-12bf-4243-8101-e337c334a88d None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79/users/9757335c28f74ea5a8fbdedc0ae7682a/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:38.791544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b317e-12bf-4243-8101-e337c334a88d None admin] RBAC: Authorizing `identity:create_grant(project_id=2ec79012680c43e3bfc414986fbb0b79, user_id=9757335c28f74ea5a8fbdedc0ae7682a, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:38.793068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b317e-12bf-4243-8101-e337c334a88d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:38.816417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 59/119] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:49:38 2022] PUT /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79/users/9757335c28f74ea5a8fbdedc0ae7682a/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:49:38.817540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b317e-12bf-4243-8101-e337c334a88d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:38.829610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780d53f-8789-4cdf-abbe-3469031ac762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:38.831147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780d53f-8789-4cdf-abbe-3469031ac762 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:38.892735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:38.943584 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 61/120] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:38.944509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780d53f-8789-4cdf-abbe-3469031ac762 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:38.951306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b317e-12bf-4243-8101-e337c334a88d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:38.992649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33673a43-df64-4457-b87b-3f716b7162f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:38.993314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33673a43-df64-4457-b87b-3f716b7162f0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:38.993602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33673a43-df64-4457-b87b-3f716b7162f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:38.993880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33673a43-df64-4457-b87b-3f716b7162f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:39.006518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33673a43-df64-4457-b87b-3f716b7162f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-33673a43-df64-4457-b87b-3f716b7162f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:39.016000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:39.016913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 60/121] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:39.017678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33673a43-df64-4457-b87b-3f716b7162f0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:39.069070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:39.082455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780d53f-8789-4cdf-abbe-3469031ac762 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:39.127392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfe216-7ecd-4f15-9fab-9819adf2a2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:39.128037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfe216-7ecd-4f15-9fab-9819adf2a2db None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:39.128381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfe216-7ecd-4f15-9fab-9819adf2a2db None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:39.128651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfe216-7ecd-4f15-9fab-9819adf2a2db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:39.143370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfe216-7ecd-4f15-9fab-9819adf2a2db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:39.143881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33673a43-df64-4457-b87b-3f716b7162f0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-e5bfe216-7ecd-4f15-9fab-9819adf2a2db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:39.149613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:39.150470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:39.151232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:39.151841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 62/122] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:49:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:39.156794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfe216-7ecd-4f15-9fab-9819adf2a2db None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:39.177372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:39.204162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4bfe3-cde1-4829-a23e-83bd9b6c734e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:39.205198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4bfe3-cde1-4829-a23e-83bd9b6c734e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:39.205542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4bfe3-cde1-4829-a23e-83bd9b6c734e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:39.205832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4bfe3-cde1-4829-a23e-83bd9b6c734e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:39.223126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4bfe3-cde1-4829-a23e-83bd9b6c734e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c6e4bfe3-cde1-4829-a23e-83bd9b6c734e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:39.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:39.229453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:39.230214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:39.232429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 61/123] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:39.233393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4bfe3-cde1-4829-a23e-83bd9b6c734e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:39.291457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfe216-7ecd-4f15-9fab-9819adf2a2db None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:39.338766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d2cae-7c63-4502-927f-d97106faf807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:39.339098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:39.340239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d2cae-7c63-4502-927f-d97106faf807 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:39.340681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d2cae-7c63-4502-927f-d97106faf807 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:39.341422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d2cae-7c63-4502-927f-d97106faf807 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:39.344416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d2cae-7c63-4502-927f-d97106faf807 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:39.346037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d2cae-7c63-4502-927f-d97106faf807 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:39.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4bfe3-cde1-4829-a23e-83bd9b6c734e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:39.348454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 63/124] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:39 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:39.349359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d2cae-7c63-4502-927f-d97106faf807 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:39.391961 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a6369-7737-4344-b0d4-e86e23700e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:39.392846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a6369-7737-4344-b0d4-e86e23700e3b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:39.393297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a6369-7737-4344-b0d4-e86e23700e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:39.393768 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a6369-7737-4344-b0d4-e86e23700e3b None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:39.394632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a6369-7737-4344-b0d4-e86e23700e3b None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:39.395081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a6369-7737-4344-b0d4-e86e23700e3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:39.397139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a6369-7737-4344-b0d4-e86e23700e3b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:39.408656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 62/125] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:39.409705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a6369-7737-4344-b0d4-e86e23700e3b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:39.441930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:39.461942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d2cae-7c63-4502-927f-d97106faf807 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:39.500959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe7a56-71a8-4a7b-8f97-ef661d032572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:39.501839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe7a56-71a8-4a7b-8f97-ef661d032572 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:39.502277 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe7a56-71a8-4a7b-8f97-ef661d032572 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:39.502552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe7a56-71a8-4a7b-8f97-ef661d032572 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:39.503231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe7a56-71a8-4a7b-8f97-ef661d032572 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:39.503537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe7a56-71a8-4a7b-8f97-ef661d032572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:39.505008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe7a56-71a8-4a7b-8f97-ef661d032572 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:39.516125 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 64/126] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:49:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:39.516977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe7a56-71a8-4a7b-8f97-ef661d032572 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:39.541484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a6369-7737-4344-b0d4-e86e23700e3b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:39.579408 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de516276-ef4f-454a-8fbd-e25e9e34c921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:39.580440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de516276-ef4f-454a-8fbd-e25e9e34c921 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:39.580935 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de516276-ef4f-454a-8fbd-e25e9e34c921 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:39.581505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de516276-ef4f-454a-8fbd-e25e9e34c921 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:39.582503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de516276-ef4f-454a-8fbd-e25e9e34c921 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:39.583012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de516276-ef4f-454a-8fbd-e25e9e34c921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:39.584770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de516276-ef4f-454a-8fbd-e25e9e34c921 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:39.593946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 63/127] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:39.595140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de516276-ef4f-454a-8fbd-e25e9e34c921 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:39.626348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe7a56-71a8-4a7b-8f97-ef661d032572 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:39.661578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b38eed-fe04-4397-8e8b-460d90650c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:39.662117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b38eed-fe04-4397-8e8b-460d90650c1f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:39.662403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b38eed-fe04-4397-8e8b-460d90650c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:39.663006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b38eed-fe04-4397-8e8b-460d90650c1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:39.663728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b38eed-fe04-4397-8e8b-460d90650c1f None admin] RBAC: Adding query filter params (role.id=7fcd054cc7c549b18e4ac97d4a405e8c, scope.project.id=2ec79012680c43e3bfc414986fbb0b79, user.id=d70bcc9f75eb448e807921f69100128f) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:39.664060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b38eed-fe04-4397-8e8b-460d90650c1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:39.665559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b38eed-fe04-4397-8e8b-460d90650c1f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:39.671048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 65/128] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:39 2022] GET /identity/v3/role_assignments?user.id=d70bcc9f75eb448e807921f69100128f&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=7fcd054cc7c549b18e4ac97d4a405e8c => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:39.672166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b38eed-fe04-4397-8e8b-460d90650c1f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:39.672626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:39.683878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:39.719431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de516276-ef4f-454a-8fbd-e25e9e34c921 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:39.759427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51eb2fe-fd4e-4325-80cc-869b7ee79351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:39.760496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eb2fe-fd4e-4325-80cc-869b7ee79351 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:39.761069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eb2fe-fd4e-4325-80cc-869b7ee79351 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:39.761524 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eb2fe-fd4e-4325-80cc-869b7ee79351 None admin] PATH_INFO: `/v3/users/7d7bdddb7df24e8a9615b6909130f4fc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:39.763045 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51eb2fe-fd4e-4325-80cc-869b7ee79351 None admin] RBAC: Authorizing `identity:get_user(user_id=7d7bdddb7df24e8a9615b6909130f4fc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:39.771373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51eb2fe-fd4e-4325-80cc-869b7ee79351 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:39.773581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 64/129] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:39 2022] GET /identity/v3/users/7d7bdddb7df24e8a9615b6909130f4fc => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:39.775037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51eb2fe-fd4e-4325-80cc-869b7ee79351 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:39.801323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b38eed-fe04-4397-8e8b-460d90650c1f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:39.877788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93c9b0-5f51-42b5-83ea-19d8411768b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:39.878436 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93c9b0-5f51-42b5-83ea-19d8411768b2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:39.878722 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93c9b0-5f51-42b5-83ea-19d8411768b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:39.879002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93c9b0-5f51-42b5-83ea-19d8411768b2 None admin] PATH_INFO: `/v3/users/7d7bdddb7df24e8a9615b6909130f4fc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:39.880879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93c9b0-5f51-42b5-83ea-19d8411768b2 None admin] RBAC: Authorizing `identity:get_user(user_id=7d7bdddb7df24e8a9615b6909130f4fc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:39.882566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93c9b0-5f51-42b5-83ea-19d8411768b2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:39.884413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 66/130] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:39 2022] GET /identity/v3/users/7d7bdddb7df24e8a9615b6909130f4fc => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:39.885386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93c9b0-5f51-42b5-83ea-19d8411768b2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:39.930485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:39.966945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51eb2fe-fd4e-4325-80cc-869b7ee79351 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:39.995896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:40.006191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d8193-678f-4311-ab54-814774500931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:40.006883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d8193-678f-4311-ab54-814774500931 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:40.007167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d8193-678f-4311-ab54-814774500931 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:40.007444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d8193-678f-4311-ab54-814774500931 None admin] PATH_INFO: `/v3/users/7d7bdddb7df24e8a9615b6909130f4fc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:40.008979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d8193-678f-4311-ab54-814774500931 None admin] RBAC: Authorizing `identity:get_user(user_id=7d7bdddb7df24e8a9615b6909130f4fc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:40.010557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d8193-678f-4311-ab54-814774500931 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:40.012523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 65/131] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:39 2022] GET /identity/v3/users/7d7bdddb7df24e8a9615b6909130f4fc => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:40.013193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d8193-678f-4311-ab54-814774500931 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:40.036345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93c9b0-5f51-42b5-83ea-19d8411768b2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:40.073243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573b767-02e1-4178-8440-ad041fa68846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:40.073888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b767-02e1-4178-8440-ad041fa68846 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:40.074170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b767-02e1-4178-8440-ad041fa68846 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:40.074446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b767-02e1-4178-8440-ad041fa68846 None admin] PATH_INFO: `/v3/projects/44fd0577657f4aa0838705dc26102320` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:40.075594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573b767-02e1-4178-8440-ad041fa68846 None admin] RBAC: Authorizing `identity:get_project(project_id=44fd0577657f4aa0838705dc26102320)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:40.077081 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573b767-02e1-4178-8440-ad041fa68846 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:40.078785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 67/132] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:39 2022] GET /identity/v3/projects/44fd0577657f4aa0838705dc26102320 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:40.079545 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573b767-02e1-4178-8440-ad041fa68846 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:40.127762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d8193-678f-4311-ab54-814774500931 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:40.165456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead8b6b-55f0-41f3-8a80-23a0cae38858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:40.166091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead8b6b-55f0-41f3-8a80-23a0cae38858 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:40.166376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead8b6b-55f0-41f3-8a80-23a0cae38858 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:40.166655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead8b6b-55f0-41f3-8a80-23a0cae38858 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:40.167772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead8b6b-55f0-41f3-8a80-23a0cae38858 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:40.169383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead8b6b-55f0-41f3-8a80-23a0cae38858 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:40.171608 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 66/133] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:40 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:40.173059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead8b6b-55f0-41f3-8a80-23a0cae38858 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:40.200129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:40.220072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573b767-02e1-4178-8440-ad041fa68846 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:40.258779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a122e-0b72-4c9e-8af5-17c4aa2d49fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:40.259417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a122e-0b72-4c9e-8af5-17c4aa2d49fc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:40.259699 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a122e-0b72-4c9e-8af5-17c4aa2d49fc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:40.259974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a122e-0b72-4c9e-8af5-17c4aa2d49fc None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:40.262256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a122e-0b72-4c9e-8af5-17c4aa2d49fc None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:40.263717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a122e-0b72-4c9e-8af5-17c4aa2d49fc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:40.265370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 68/134] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:40 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:40.266152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a122e-0b72-4c9e-8af5-17c4aa2d49fc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:40.297924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:40.385541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a122e-0b72-4c9e-8af5-17c4aa2d49fc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:40.398257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead8b6b-55f0-41f3-8a80-23a0cae38858 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:40.422788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e5301-2ed1-44dc-8d48-815726f3fcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:40.423457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e5301-2ed1-44dc-8d48-815726f3fcbf None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:40.423739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e5301-2ed1-44dc-8d48-815726f3fcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:40.424014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e5301-2ed1-44dc-8d48-815726f3fcbf None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79/users/7d7bdddb7df24e8a9615b6909130f4fc/roles/fc1186ecd0f74b2fad8155d528f268f9` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:40.426209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e5301-2ed1-44dc-8d48-815726f3fcbf None admin] RBAC: Authorizing `identity:create_grant(project_id=2ec79012680c43e3bfc414986fbb0b79, user_id=7d7bdddb7df24e8a9615b6909130f4fc, role_id=fc1186ecd0f74b2fad8155d528f268f9)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:40.427682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e5301-2ed1-44dc-8d48-815726f3fcbf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:40.444097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 69/135] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:49:40 2022] PUT /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79/users/7d7bdddb7df24e8a9615b6909130f4fc/roles/fc1186ecd0f74b2fad8155d528f268f9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:49:40.445037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e5301-2ed1-44dc-8d48-815726f3fcbf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:40.496865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8a606-ee23-44bc-b72a-464b0e2bf619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:40.497465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8a606-ee23-44bc-b72a-464b0e2bf619 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:40.497750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8a606-ee23-44bc-b72a-464b0e2bf619 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:40.498026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8a606-ee23-44bc-b72a-464b0e2bf619 None admin] PATH_INFO: `/v3/projects/44fd0577657f4aa0838705dc26102320/users/7d7bdddb7df24e8a9615b6909130f4fc/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:40.505578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8a606-ee23-44bc-b72a-464b0e2bf619 None admin] RBAC: Authorizing `identity:create_grant(project_id=44fd0577657f4aa0838705dc26102320, user_id=7d7bdddb7df24e8a9615b6909130f4fc, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:40.507079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8a606-ee23-44bc-b72a-464b0e2bf619 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:40.538610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 67/136] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:49:40 2022] PUT /identity/v3/projects/44fd0577657f4aa0838705dc26102320/users/7d7bdddb7df24e8a9615b6909130f4fc/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:49:40.540067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803c7b7-9140-4d02-8409-6d39c1d0e7ef None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:40.552725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803c7b7-9140-4d02-8409-6d39c1d0e7ef None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:40.553050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803c7b7-9140-4d02-8409-6d39c1d0e7ef None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:40.554888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 68/137] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:40 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:40.584428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:40.590057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1f5d0-b759-49c7-b542-66431ec6aee7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:40.590365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1f5d0-b759-49c7-b542-66431ec6aee7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:40.590623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1f5d0-b759-49c7-b542-66431ec6aee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:40.634216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e5301-2ed1-44dc-8d48-815726f3fcbf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:40.637315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-03b1f5d0-b759-49c7-b542-66431ec6aee7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:40.638632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-03b1f5d0-b759-49c7-b542-66431ec6aee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:40.685234 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5460d9d-b701-4816-a7d8-a09817ff0f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:40.685928 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5460d9d-b701-4816-a7d8-a09817ff0f75 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:40.686205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5460d9d-b701-4816-a7d8-a09817ff0f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:40.686479 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5460d9d-b701-4816-a7d8-a09817ff0f75 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79/users/7d7bdddb7df24e8a9615b6909130f4fc/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:40.688552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5460d9d-b701-4816-a7d8-a09817ff0f75 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ec79012680c43e3bfc414986fbb0b79, user_id=7d7bdddb7df24e8a9615b6909130f4fc, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:40.690733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5460d9d-b701-4816-a7d8-a09817ff0f75 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:40.709240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 70/138] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:49:40 2022] PUT /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79/users/7d7bdddb7df24e8a9615b6909130f4fc/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:49:40.798593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 69/139] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:41.099012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639f59a-afed-439c-b6ed-f10ea65dea04 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:41.099485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639f59a-afed-439c-b6ed-f10ea65dea04 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:41.099558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639f59a-afed-439c-b6ed-f10ea65dea04 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:41.101079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 71/140] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:41.116876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950be65-ebfd-46bc-990a-80115b0d474a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:41.117155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950be65-ebfd-46bc-990a-80115b0d474a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:41.117481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950be65-ebfd-46bc-990a-80115b0d474a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:41.142337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f950be65-ebfd-46bc-990a-80115b0d474a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:41.143520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f950be65-ebfd-46bc-990a-80115b0d474a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:41.154087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:41.183448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639f59a-afed-439c-b6ed-f10ea65dea04 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:41.257407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 70/141] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:41.311808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639f59a-afed-439c-b6ed-f10ea65dea04 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:41.344364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:41.362107 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ad710-b133-47a8-a1a1-14d52c77beff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:41.362658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ad710-b133-47a8-a1a1-14d52c77beff None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:41.362937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ad710-b133-47a8-a1a1-14d52c77beff None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:41.363204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ad710-b133-47a8-a1a1-14d52c77beff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:41.364401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ad710-b133-47a8-a1a1-14d52c77beff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:41.365925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ad710-b133-47a8-a1a1-14d52c77beff None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:41.367818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 72/142] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:41.373428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950be65-ebfd-46bc-990a-80115b0d474a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:41.482728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950be65-ebfd-46bc-990a-80115b0d474a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:41.519995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2638fe-59c5-4d80-b055-ecedc05b9a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:41.520637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2638fe-59c5-4d80-b055-ecedc05b9a97 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:41.520919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2638fe-59c5-4d80-b055-ecedc05b9a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:41.521193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2638fe-59c5-4d80-b055-ecedc05b9a97 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:41.521857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2638fe-59c5-4d80-b055-ecedc05b9a97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:41.523310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2638fe-59c5-4d80-b055-ecedc05b9a97 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:41.541306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:41.551062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 71/143] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:49:41 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:49:41.781226 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ad710-b133-47a8-a1a1-14d52c77beff None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:41.895426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ad710-b133-47a8-a1a1-14d52c77beff None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:41.931259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba307e3-794d-4fef-a8f5-c7e0a8257a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:41.931886 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba307e3-794d-4fef-a8f5-c7e0a8257a51 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:41.932193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba307e3-794d-4fef-a8f5-c7e0a8257a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:41.932467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba307e3-794d-4fef-a8f5-c7e0a8257a51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:41.942712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba307e3-794d-4fef-a8f5-c7e0a8257a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-5ba307e3-794d-4fef-a8f5-c7e0a8257a51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:41.947759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:41.949498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:41.950602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 73/144] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:49:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:41.954807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2638fe-59c5-4d80-b055-ecedc05b9a97 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:42.084977 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2638fe-59c5-4d80-b055-ecedc05b9a97 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:42.130188 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b27fb1-8568-4540-98c9-4215b2352fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:42.130807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b27fb1-8568-4540-98c9-4215b2352fff None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:42.131090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b27fb1-8568-4540-98c9-4215b2352fff None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:42.131356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b27fb1-8568-4540-98c9-4215b2352fff None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:42.132024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b27fb1-8568-4540-98c9-4215b2352fff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:42.132351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b27fb1-8568-4540-98c9-4215b2352fff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:42.133743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b27fb1-8568-4540-98c9-4215b2352fff None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:42.148318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 72/145] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:42.153860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba307e3-794d-4fef-a8f5-c7e0a8257a51 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:42.230774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:42.292577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba307e3-794d-4fef-a8f5-c7e0a8257a51 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:42.335625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7cf80-4380-46a5-8e6b-865f93e5ce31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:42.336754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7cf80-4380-46a5-8e6b-865f93e5ce31 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:42.337503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7cf80-4380-46a5-8e6b-865f93e5ce31 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:42.338011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7cf80-4380-46a5-8e6b-865f93e5ce31 None admin] PATH_INFO: `/v3/users/9757335c28f74ea5a8fbdedc0ae7682a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:42.339477 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7cf80-4380-46a5-8e6b-865f93e5ce31 None admin] RBAC: Authorizing `identity:get_user(user_id=9757335c28f74ea5a8fbdedc0ae7682a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:42.341205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7cf80-4380-46a5-8e6b-865f93e5ce31 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:42.343070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 74/146] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:42 2022] GET /identity/v3/users/9757335c28f74ea5a8fbdedc0ae7682a => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:42.349541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b27fb1-8568-4540-98c9-4215b2352fff None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:42.463025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:42.527252 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b27fb1-8568-4540-98c9-4215b2352fff None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:42.554194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af74c3-7116-4a7a-9825-941d116b0ddf None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:42.554501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af74c3-7116-4a7a-9825-941d116b0ddf None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:42.554745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af74c3-7116-4a7a-9825-941d116b0ddf None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:42.556269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 75/147] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:42.561781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb60ce4-c2e2-4e6b-84e7-81e944329aad None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:42.562070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb60ce4-c2e2-4e6b-84e7-81e944329aad None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:42.562347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb60ce4-c2e2-4e6b-84e7-81e944329aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:42.567195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27fa91-6024-48f1-b22b-fb2fb9b7c4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:42.567808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27fa91-6024-48f1-b22b-fb2fb9b7c4ed None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:42.568122 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27fa91-6024-48f1-b22b-fb2fb9b7c4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:42.568398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27fa91-6024-48f1-b22b-fb2fb9b7c4ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:42.598661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5cb60ce4-c2e2-4e6b-84e7-81e944329aad None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:42.600079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5cb60ce4-c2e2-4e6b-84e7-81e944329aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:42.684215 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 76/148] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:42.712852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27fa91-6024-48f1-b22b-fb2fb9b7c4ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:42.714540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27fa91-6024-48f1-b22b-fb2fb9b7c4ed None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:42.777338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:42.822278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 73/149] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:42.828254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb60ce4-c2e2-4e6b-84e7-81e944329aad None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:42.905443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:42.953188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb60ce4-c2e2-4e6b-84e7-81e944329aad None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:42.990890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fc2d5-f4f0-4651-af78-6d0231692257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:42.991915 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fc2d5-f4f0-4651-af78-6d0231692257 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:42.992444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fc2d5-f4f0-4651-af78-6d0231692257 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:42.992929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fc2d5-f4f0-4651-af78-6d0231692257 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:42.994299 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fc2d5-f4f0-4651-af78-6d0231692257 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:42.995978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fc2d5-f4f0-4651-af78-6d0231692257 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:42.998230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 77/150] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:42 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:43.009255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27fa91-6024-48f1-b22b-fb2fb9b7c4ed None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:43.092026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fc2d5-f4f0-4651-af78-6d0231692257 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:43.134969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27fa91-6024-48f1-b22b-fb2fb9b7c4ed None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:43.181889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8c0e7-aa49-4deb-a1db-6e2994b7e938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:43.182514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8c0e7-aa49-4deb-a1db-6e2994b7e938 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:43.182795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8c0e7-aa49-4deb-a1db-6e2994b7e938 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:43.183068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8c0e7-aa49-4deb-a1db-6e2994b7e938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:43.183809 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8c0e7-aa49-4deb-a1db-6e2994b7e938 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=2ec79012680c43e3bfc414986fbb0b79, user.id=9757335c28f74ea5a8fbdedc0ae7682a) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:43.184181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8c0e7-aa49-4deb-a1db-6e2994b7e938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:43.185654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8c0e7-aa49-4deb-a1db-6e2994b7e938 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:43.204750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 74/151] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:43 2022] GET /identity/v3/role_assignments?user.id=9757335c28f74ea5a8fbdedc0ae7682a&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:43.292615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fc2d5-f4f0-4651-af78-6d0231692257 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:43.334068 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:43.339642 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb898ac0-d1ab-4d2b-8aa8-b283e969f383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:43.340722 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb898ac0-d1ab-4d2b-8aa8-b283e969f383 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:43.341454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb898ac0-d1ab-4d2b-8aa8-b283e969f383 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:43.341734 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb898ac0-d1ab-4d2b-8aa8-b283e969f383 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:43.358137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb898ac0-d1ab-4d2b-8aa8-b283e969f383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-cb898ac0-d1ab-4d2b-8aa8-b283e969f383 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:43.363011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:43.363941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:43.364822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 78/152] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:43.371682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8c0e7-aa49-4deb-a1db-6e2994b7e938 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:43.401045 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:43.491398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8c0e7-aa49-4deb-a1db-6e2994b7e938 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:43.533995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47136f48-99f7-4707-b588-7f687accb37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47136f48-99f7-4707-b588-7f687accb37c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:43.534916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47136f48-99f7-4707-b588-7f687accb37c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:43.535189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47136f48-99f7-4707-b588-7f687accb37c None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:43.535870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47136f48-99f7-4707-b588-7f687accb37c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:43.536202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47136f48-99f7-4707-b588-7f687accb37c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:43.537610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47136f48-99f7-4707-b588-7f687accb37c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:43.548056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 75/153] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:43.553398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb898ac0-d1ab-4d2b-8aa8-b283e969f383 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:43.662583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb898ac0-d1ab-4d2b-8aa8-b283e969f383 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:43.703136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05006f8e-2db1-447e-b244-71d352cfdde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:43.704343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05006f8e-2db1-447e-b244-71d352cfdde8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:43.704629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05006f8e-2db1-447e-b244-71d352cfdde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:43.704902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05006f8e-2db1-447e-b244-71d352cfdde8 None admin] PATH_INFO: `/v3/users/d70bcc9f75eb448e807921f69100128f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:43.706073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05006f8e-2db1-447e-b244-71d352cfdde8 None admin] RBAC: Authorizing `identity:get_user(user_id=d70bcc9f75eb448e807921f69100128f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:43.707556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05006f8e-2db1-447e-b244-71d352cfdde8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:43.709569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 79/154] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:43 2022] GET /identity/v3/users/d70bcc9f75eb448e807921f69100128f => generated 289 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:43.715878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47136f48-99f7-4707-b588-7f687accb37c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:43.809984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:43.830053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47136f48-99f7-4707-b588-7f687accb37c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:43.856308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27432c-5a43-4836-88d6-8d96651197c6 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:43.856620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27432c-5a43-4836-88d6-8d96651197c6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:43.856846 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27432c-5a43-4836-88d6-8d96651197c6 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:43.858358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 80/155] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:43.863637 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80325c8f-0508-4a4d-8b04-355afdba02c2 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:43.863920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80325c8f-0508-4a4d-8b04-355afdba02c2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:43.864252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80325c8f-0508-4a4d-8b04-355afdba02c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:43.901158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-80325c8f-0508-4a4d-8b04-355afdba02c2 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:43.902874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-80325c8f-0508-4a4d-8b04-355afdba02c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:43.909980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc860e-5b1d-44b5-8f52-7974ed2cb197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:43.911524 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc860e-5b1d-44b5-8f52-7974ed2cb197 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:43.911827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc860e-5b1d-44b5-8f52-7974ed2cb197 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:43.912126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc860e-5b1d-44b5-8f52-7974ed2cb197 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:43.913278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc860e-5b1d-44b5-8f52-7974ed2cb197 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:43.915073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc860e-5b1d-44b5-8f52-7974ed2cb197 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:43.916937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 76/156] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:43 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:43.922312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc860e-5b1d-44b5-8f52-7974ed2cb197 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:43.995740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:44.029659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 81/157] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:44.054143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc860e-5b1d-44b5-8f52-7974ed2cb197 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:44.095619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444de00-2985-4c6a-9e31-dbea32101154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:44.096322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444de00-2985-4c6a-9e31-dbea32101154 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:44.096607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444de00-2985-4c6a-9e31-dbea32101154 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:44.096883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444de00-2985-4c6a-9e31-dbea32101154 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79/users/d70bcc9f75eb448e807921f69100128f/roles/7fcd054cc7c549b18e4ac97d4a405e8c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:44.098826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444de00-2985-4c6a-9e31-dbea32101154 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ec79012680c43e3bfc414986fbb0b79, user_id=d70bcc9f75eb448e807921f69100128f, role_id=7fcd054cc7c549b18e4ac97d4a405e8c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:44.100312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444de00-2985-4c6a-9e31-dbea32101154 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:44.118189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 77/158] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:49:43 2022] PUT /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79/users/d70bcc9f75eb448e807921f69100128f/roles/7fcd054cc7c549b18e4ac97d4a405e8c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:49:44.128419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f1e5-c045-4d82-b900-5c5b37bc0480 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:44.129072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f1e5-c045-4d82-b900-5c5b37bc0480 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:44.129560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f1e5-c045-4d82-b900-5c5b37bc0480 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:44.131512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 82/159] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:44.138105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f159a-0928-4752-b7ea-e1c1ae509a3d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:44.138457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f159a-0928-4752-b7ea-e1c1ae509a3d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:44.138738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f159a-0928-4752-b7ea-e1c1ae509a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:44.158694 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-653f159a-0928-4752-b7ea-e1c1ae509a3d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:44.159946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-653f159a-0928-4752-b7ea-e1c1ae509a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:44.171189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:44.217342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f002e-b796-476d-9061-45cadbf0161b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:44.217965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f002e-b796-476d-9061-45cadbf0161b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:44.218476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f002e-b796-476d-9061-45cadbf0161b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:44.221489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 83/160] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:44.227897 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542c9a7-9e63-45d9-a2ff-f070b49c5035 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:44.228650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542c9a7-9e63-45d9-a2ff-f070b49c5035 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:44.229151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542c9a7-9e63-45d9-a2ff-f070b49c5035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:44.263030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 78/161] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:44.269130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3542c9a7-9e63-45d9-a2ff-f070b49c5035 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:44.270375 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3542c9a7-9e63-45d9-a2ff-f070b49c5035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:44.374155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 84/162] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:44.536815 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f159a-0928-4752-b7ea-e1c1ae509a3d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:44.667803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f159a-0928-4752-b7ea-e1c1ae509a3d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:44.711017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542c9a7-9e63-45d9-a2ff-f070b49c5035 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:44.721853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d09fd-9a85-4230-86ee-2072b1843318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:44.722501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d09fd-9a85-4230-86ee-2072b1843318 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:44.722777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d09fd-9a85-4230-86ee-2072b1843318 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:44.723046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d09fd-9a85-4230-86ee-2072b1843318 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:44.741219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d09fd-9a85-4230-86ee-2072b1843318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7a3d09fd-9a85-4230-86ee-2072b1843318 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:44.745972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:44.746824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:44.747629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 79/163] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:44.748373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d09fd-9a85-4230-86ee-2072b1843318 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:44.776778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:44.918717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d09fd-9a85-4230-86ee-2072b1843318 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:44.979051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a07bc-d89f-43b1-9629-33c163f3ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:44.979695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a07bc-d89f-43b1-9629-33c163f3ecda None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:44.979972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a07bc-d89f-43b1-9629-33c163f3ecda None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:44.980267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a07bc-d89f-43b1-9629-33c163f3ecda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:44.982938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542c9a7-9e63-45d9-a2ff-f070b49c5035 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:45.013268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a07bc-d89f-43b1-9629-33c163f3ecda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:45.021624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-711a07bc-d89f-43b1-9629-33c163f3ecda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:45.022866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:45.023900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 80/164] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:49:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:45.024771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a07bc-d89f-43b1-9629-33c163f3ecda None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:45.076939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8eb9cd-4e36-42cf-8d3f-cd76c1683831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:45.077581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8eb9cd-4e36-42cf-8d3f-cd76c1683831 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:45.077862 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8eb9cd-4e36-42cf-8d3f-cd76c1683831 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:45.078135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8eb9cd-4e36-42cf-8d3f-cd76c1683831 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:45.098658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8eb9cd-4e36-42cf-8d3f-cd76c1683831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-fa8eb9cd-4e36-42cf-8d3f-cd76c1683831 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:45.103894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:45.104968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:45.105756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:45.106359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:45.106752 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 85/165] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:45.106752 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8eb9cd-4e36-42cf-8d3f-cd76c1683831 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:45.169259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a07bc-d89f-43b1-9629-33c163f3ecda None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:45.225316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85910ca4-24f1-4920-b311-e6ccabea88a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:45.225934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910ca4-24f1-4920-b311-e6ccabea88a6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:45.226216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910ca4-24f1-4920-b311-e6ccabea88a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:45.226492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910ca4-24f1-4920-b311-e6ccabea88a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:45.227168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85910ca4-24f1-4920-b311-e6ccabea88a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:45.227483 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85910ca4-24f1-4920-b311-e6ccabea88a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:45.228907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85910ca4-24f1-4920-b311-e6ccabea88a6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:45.246600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 81/166] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:45.247300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85910ca4-24f1-4920-b311-e6ccabea88a6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:45.315819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8eb9cd-4e36-42cf-8d3f-cd76c1683831 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:45.363321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85910ca4-24f1-4920-b311-e6ccabea88a6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:45.381794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad520a7-2dea-482c-90b4-e2bb561b3a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:45.382455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad520a7-2dea-482c-90b4-e2bb561b3a0a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:45.382681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad520a7-2dea-482c-90b4-e2bb561b3a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:45.382951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad520a7-2dea-482c-90b4-e2bb561b3a0a None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:45.383713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad520a7-2dea-482c-90b4-e2bb561b3a0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:45.384026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad520a7-2dea-482c-90b4-e2bb561b3a0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:45.385461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad520a7-2dea-482c-90b4-e2bb561b3a0a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:45.392488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:45.402112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 86/167] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:49:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:45.403053 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad520a7-2dea-482c-90b4-e2bb561b3a0a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:45.410139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e8ef9-268d-4a94-9cff-df280415b3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:45.410683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e8ef9-268d-4a94-9cff-df280415b3eb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:45.410960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e8ef9-268d-4a94-9cff-df280415b3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:45.411234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e8ef9-268d-4a94-9cff-df280415b3eb None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:45.411913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e8ef9-268d-4a94-9cff-df280415b3eb None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:45.412241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e8ef9-268d-4a94-9cff-df280415b3eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:45.413877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e8ef9-268d-4a94-9cff-df280415b3eb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:45.423654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 82/168] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:45.424453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e8ef9-268d-4a94-9cff-df280415b3eb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:45.448733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:45.543405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad520a7-2dea-482c-90b4-e2bb561b3a0a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:45.611221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976340e-175b-4df1-9881-4b8a2f85eaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:45.612302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976340e-175b-4df1-9881-4b8a2f85eaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:45.612924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976340e-175b-4df1-9881-4b8a2f85eaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:45.613400 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976340e-175b-4df1-9881-4b8a2f85eaf2 None admin] PATH_INFO: `/v3/users/7d7bdddb7df24e8a9615b6909130f4fc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:45.614739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976340e-175b-4df1-9881-4b8a2f85eaf2 None admin] RBAC: Authorizing `identity:get_user(user_id=7d7bdddb7df24e8a9615b6909130f4fc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:45.620143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e8ef9-268d-4a94-9cff-df280415b3eb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:45.621298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976340e-175b-4df1-9881-4b8a2f85eaf2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:45.623236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 87/169] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:45 2022] GET /identity/v3/users/7d7bdddb7df24e8a9615b6909130f4fc => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:45.624231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976340e-175b-4df1-9881-4b8a2f85eaf2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:45.689272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a7ae0-5508-43a1-a235-c1ab2d5c79b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:45.691901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a7ae0-5508-43a1-a235-c1ab2d5c79b7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:45.692413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a7ae0-5508-43a1-a235-c1ab2d5c79b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:45.692890 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a7ae0-5508-43a1-a235-c1ab2d5c79b7 None admin] PATH_INFO: `/v3/users/7d7bdddb7df24e8a9615b6909130f4fc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:45.694308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a7ae0-5508-43a1-a235-c1ab2d5c79b7 None admin] RBAC: Authorizing `identity:get_user(user_id=7d7bdddb7df24e8a9615b6909130f4fc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:45.696092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a7ae0-5508-43a1-a235-c1ab2d5c79b7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:45.698111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 83/170] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:45 2022] GET /identity/v3/users/7d7bdddb7df24e8a9615b6909130f4fc => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:45.699098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a7ae0-5508-43a1-a235-c1ab2d5c79b7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:45.766087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:45.817049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a7ae0-5508-43a1-a235-c1ab2d5c79b7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:45.826230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976340e-175b-4df1-9881-4b8a2f85eaf2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:45.828651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:45.857669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62929ff4-5035-4e9a-a5e9-593bd2a7987d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:45.858858 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62929ff4-5035-4e9a-a5e9-593bd2a7987d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:45.859343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62929ff4-5035-4e9a-a5e9-593bd2a7987d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:45.859816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62929ff4-5035-4e9a-a5e9-593bd2a7987d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:45.888851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbfa46-341a-4931-bbe3-e760adec2cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:45.889460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbfa46-341a-4931-bbe3-e760adec2cbe None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:45.889695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbfa46-341a-4931-bbe3-e760adec2cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:45.889972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbfa46-341a-4931-bbe3-e760adec2cbe None admin] PATH_INFO: `/v3/users/7d7bdddb7df24e8a9615b6909130f4fc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:45.891150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbfa46-341a-4931-bbe3-e760adec2cbe None admin] RBAC: Authorizing `identity:get_user(user_id=7d7bdddb7df24e8a9615b6909130f4fc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:45.893265 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbfa46-341a-4931-bbe3-e760adec2cbe None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:45.895169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 88/171] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:45 2022] GET /identity/v3/users/7d7bdddb7df24e8a9615b6909130f4fc => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:45.896329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac1dec-dd3e-4560-8056-989a88859048 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:45.896810 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac1dec-dd3e-4560-8056-989a88859048 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:45.896914 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac1dec-dd3e-4560-8056-989a88859048 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:45.898097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 89/172] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:45.898788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac1dec-dd3e-4560-8056-989a88859048 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:45.939519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62929ff4-5035-4e9a-a5e9-593bd2a7987d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:45.941535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62929ff4-5035-4e9a-a5e9-593bd2a7987d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:46.002529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:46.048766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac1dec-dd3e-4560-8056-989a88859048 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:46.054391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 84/173] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:46.055868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60989503-331f-4830-8f2d-566263d78541 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:46.061030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60989503-331f-4830-8f2d-566263d78541 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:46.061543 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60989503-331f-4830-8f2d-566263d78541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:46.081321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-60989503-331f-4830-8f2d-566263d78541 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:46.082973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-60989503-331f-4830-8f2d-566263d78541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:46.141964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d9b2f-5d54-4724-bf07-9958a109555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:46.142590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d9b2f-5d54-4724-bf07-9958a109555a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:46.142874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d9b2f-5d54-4724-bf07-9958a109555a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:46.143155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d9b2f-5d54-4724-bf07-9958a109555a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:46.156751 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 85/174] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:46.159199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60989503-331f-4830-8f2d-566263d78541 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:46.238122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:46.266579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:46.291080 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60989503-331f-4830-8f2d-566263d78541 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:46.296138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d9b2f-5d54-4724-bf07-9958a109555a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:46.297663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d9b2f-5d54-4724-bf07-9958a109555a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:46.366988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f405710-4961-46bb-947d-20845f9ba9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:46.367609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f405710-4961-46bb-947d-20845f9ba9bb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:46.367889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f405710-4961-46bb-947d-20845f9ba9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:46.368192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f405710-4961-46bb-947d-20845f9ba9bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:46.453354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 90/175] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:46.454240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d9b2f-5d54-4724-bf07-9958a109555a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:46.477711 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f405710-4961-46bb-947d-20845f9ba9bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:46.479672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f405710-4961-46bb-947d-20845f9ba9bb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:46.533193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:46.568008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d9b2f-5d54-4724-bf07-9958a109555a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:46.580433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:46.597015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 86/176] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:46.598675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac150b87-6826-403f-9aae-00e035878da1 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:46.598963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac150b87-6826-403f-9aae-00e035878da1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:46.599235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac150b87-6826-403f-9aae-00e035878da1 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:46.600652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 87/177] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:46.605856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f7d4c-06c8-4c95-8516-570ac29b97b0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:46.606290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f7d4c-06c8-4c95-8516-570ac29b97b0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:46.606707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f7d4c-06c8-4c95-8516-570ac29b97b0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:46.612995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 88/178] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:46.612995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f7d4c-06c8-4c95-8516-570ac29b97b0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:46.619500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8be581-f4f9-4ed3-b8d3-13cb833713e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:46.620006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8be581-f4f9-4ed3-b8d3-13cb833713e0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:46.620326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8be581-f4f9-4ed3-b8d3-13cb833713e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:46.620595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8be581-f4f9-4ed3-b8d3-13cb833713e0 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:46.621737 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8be581-f4f9-4ed3-b8d3-13cb833713e0 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:46.623192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8be581-f4f9-4ed3-b8d3-13cb833713e0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:46.625000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 91/179] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:46 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:46.625680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8be581-f4f9-4ed3-b8d3-13cb833713e0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:46.734111 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8be581-f4f9-4ed3-b8d3-13cb833713e0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:46.740644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f7d4c-06c8-4c95-8516-570ac29b97b0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:46.772619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298e61d-35c8-452b-98bc-7c26b60481be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:46.773255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298e61d-35c8-452b-98bc-7c26b60481be None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:46.773537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298e61d-35c8-452b-98bc-7c26b60481be None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:46.773811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298e61d-35c8-452b-98bc-7c26b60481be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:46.779915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f20c35-e7b7-4bae-a2ed-f3a5cf807a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:46.780595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f20c35-e7b7-4bae-a2ed-f3a5cf807a75 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:46.780888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f20c35-e7b7-4bae-a2ed-f3a5cf807a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:46.781138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f20c35-e7b7-4bae-a2ed-f3a5cf807a75 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:46.782058 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:46.782900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f20c35-e7b7-4bae-a2ed-f3a5cf807a75 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:46.784418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f20c35-e7b7-4bae-a2ed-f3a5cf807a75 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:46.787341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 89/180] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:46 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:46.787341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f20c35-e7b7-4bae-a2ed-f3a5cf807a75 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:46.792528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298e61d-35c8-452b-98bc-7c26b60481be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c298e61d-35c8-452b-98bc-7c26b60481be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:46.798171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:46.799236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:46.801050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 92/181] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:49:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:46.802465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efbc35-29d5-4413-9405-195bc8eef60b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:46.802851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efbc35-29d5-4413-9405-195bc8eef60b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:46.803345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efbc35-29d5-4413-9405-195bc8eef60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:46.821598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e1efbc35-29d5-4413-9405-195bc8eef60b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:46.822811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e1efbc35-29d5-4413-9405-195bc8eef60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:46.853704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:46.897019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 93/182] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:46.898270 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593851a4-ddc4-4d82-8d9d-b166b55d9ef3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:46.898554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593851a4-ddc4-4d82-8d9d-b166b55d9ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:46.898826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593851a4-ddc4-4d82-8d9d-b166b55d9ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:46.912532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f20c35-e7b7-4bae-a2ed-f3a5cf807a75 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:46.920890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-593851a4-ddc4-4d82-8d9d-b166b55d9ef3 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:46.922618 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-593851a4-ddc4-4d82-8d9d-b166b55d9ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:46.960349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f23a0-0a55-435d-917a-ac2c85d11a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:46.961807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f23a0-0a55-435d-917a-ac2c85d11a83 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:46.962358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f23a0-0a55-435d-917a-ac2c85d11a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:46.962890 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f23a0-0a55-435d-917a-ac2c85d11a83 None admin] PATH_INFO: `/v3/projects/44fd0577657f4aa0838705dc26102320` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:46.964454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f23a0-0a55-435d-917a-ac2c85d11a83 None admin] RBAC: Authorizing `identity:get_project(project_id=44fd0577657f4aa0838705dc26102320)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:46.966318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f23a0-0a55-435d-917a-ac2c85d11a83 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:46.969325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 90/183] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:46 2022] GET /identity/v3/projects/44fd0577657f4aa0838705dc26102320 => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:46.970239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f23a0-0a55-435d-917a-ac2c85d11a83 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:47.011763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 94/184] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:47.011763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593851a4-ddc4-4d82-8d9d-b166b55d9ef3 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:47.106758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:47.114815 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f23a0-0a55-435d-917a-ac2c85d11a83 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:47.140541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:47.154167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274dd676-9373-48a1-aa7b-38e5dc85c732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:47.154493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593851a4-ddc4-4d82-8d9d-b166b55d9ef3 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:47.154777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274dd676-9373-48a1-aa7b-38e5dc85c732 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:47.155063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274dd676-9373-48a1-aa7b-38e5dc85c732 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:47.155339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274dd676-9373-48a1-aa7b-38e5dc85c732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:47.156332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274dd676-9373-48a1-aa7b-38e5dc85c732 None admin] RBAC: Adding query filter params (role.id=7ab2687826544b8e80c709d6061162d4, scope.project.id=2ec79012680c43e3bfc414986fbb0b79, user.id=7d7bdddb7df24e8a9615b6909130f4fc) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:47.158008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274dd676-9373-48a1-aa7b-38e5dc85c732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:47.159485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274dd676-9373-48a1-aa7b-38e5dc85c732 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:47.166827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 91/185] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:46 2022] GET /identity/v3/role_assignments?user.id=7d7bdddb7df24e8a9615b6909130f4fc&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=7ab2687826544b8e80c709d6061162d4 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:47.168629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274dd676-9373-48a1-aa7b-38e5dc85c732 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:47.212519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c03473-fba4-447f-8af1-42d616377eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:47.213409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c03473-fba4-447f-8af1-42d616377eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:47.213692 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c03473-fba4-447f-8af1-42d616377eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:47.214033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c03473-fba4-447f-8af1-42d616377eb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:47.215239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c03473-fba4-447f-8af1-42d616377eb3 None admin] RBAC: Adding query filter params (role.id=fc1186ecd0f74b2fad8155d528f268f9, scope.project.id=2ec79012680c43e3bfc414986fbb0b79, user.id=7d7bdddb7df24e8a9615b6909130f4fc) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:47.215778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c03473-fba4-447f-8af1-42d616377eb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:47.218429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c03473-fba4-447f-8af1-42d616377eb3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:47.227691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 95/186] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:47 2022] GET /identity/v3/role_assignments?user.id=7d7bdddb7df24e8a9615b6909130f4fc&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=fc1186ecd0f74b2fad8155d528f268f9 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:47.228696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c03473-fba4-447f-8af1-42d616377eb3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:47.296165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274dd676-9373-48a1-aa7b-38e5dc85c732 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:47.339975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81863b-2999-4bae-9f19-c2f905cbd148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:47.341020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81863b-2999-4bae-9f19-c2f905cbd148 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:47.341482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81863b-2999-4bae-9f19-c2f905cbd148 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:47.342005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81863b-2999-4bae-9f19-c2f905cbd148 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:47.343021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81863b-2999-4bae-9f19-c2f905cbd148 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:47.343545 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81863b-2999-4bae-9f19-c2f905cbd148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:47.345821 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81863b-2999-4bae-9f19-c2f905cbd148 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:47.356420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 92/187] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:47.358386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58818-862f-4f5d-9b8f-a4d3ad59dd84 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:47.358895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58818-862f-4f5d-9b8f-a4d3ad59dd84 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:47.359363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58818-862f-4f5d-9b8f-a4d3ad59dd84 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:47.361126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 93/188] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:47.362334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be58818-862f-4f5d-9b8f-a4d3ad59dd84 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:47.369783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c03473-fba4-447f-8af1-42d616377eb3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:47.397204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:47.415438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563404a-3571-47e7-9eb1-f22fee0feb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:47.417257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563404a-3571-47e7-9eb1-f22fee0feb93 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:47.417257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563404a-3571-47e7-9eb1-f22fee0feb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:47.417257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563404a-3571-47e7-9eb1-f22fee0feb93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:47.417505 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563404a-3571-47e7-9eb1-f22fee0feb93 None admin] RBAC: Adding query filter params (role.id=7ab2687826544b8e80c709d6061162d4, scope.project.id=44fd0577657f4aa0838705dc26102320, user.id=7d7bdddb7df24e8a9615b6909130f4fc) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:47.417747 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563404a-3571-47e7-9eb1-f22fee0feb93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:47.419253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563404a-3571-47e7-9eb1-f22fee0feb93 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:47.429941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 96/189] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:47 2022] GET /identity/v3/role_assignments?user.id=7d7bdddb7df24e8a9615b6909130f4fc&scope.project.id=44fd0577657f4aa0838705dc26102320&role.id=7ab2687826544b8e80c709d6061162d4 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:47.431064 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563404a-3571-47e7-9eb1-f22fee0feb93 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:47.441645 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:47.517483 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be58818-862f-4f5d-9b8f-a4d3ad59dd84 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:47.569835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe8d2a-52e6-4899-9ad6-895eaa65739f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:47.570208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563404a-3571-47e7-9eb1-f22fee0feb93 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:47.570776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe8d2a-52e6-4899-9ad6-895eaa65739f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:47.571062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe8d2a-52e6-4899-9ad6-895eaa65739f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:47.571332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe8d2a-52e6-4899-9ad6-895eaa65739f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:47.586345 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe8d2a-52e6-4899-9ad6-895eaa65739f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7bbe8d2a-52e6-4899-9ad6-895eaa65739f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:47.593335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:47.594272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:47.595111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 94/190] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:49:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:47.600419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe8d2a-52e6-4899-9ad6-895eaa65739f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:47.610866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6ea71-5c7f-40a6-b646-7883a975bf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:47.611618 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ea71-5c7f-40a6-b646-7883a975bf2f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:47.611905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ea71-5c7f-40a6-b646-7883a975bf2f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:47.612894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ea71-5c7f-40a6-b646-7883a975bf2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:47.614062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6ea71-5c7f-40a6-b646-7883a975bf2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:47.615529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6ea71-5c7f-40a6-b646-7883a975bf2f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:47.617339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 97/191] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:47.621374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436023fa-7734-4425-a566-e6a39ef7cee1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:47.621724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436023fa-7734-4425-a566-e6a39ef7cee1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:47.622002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436023fa-7734-4425-a566-e6a39ef7cee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:47.642012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-436023fa-7734-4425-a566-e6a39ef7cee1 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:47.643207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-436023fa-7734-4425-a566-e6a39ef7cee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:47.700593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:47.715116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe8d2a-52e6-4899-9ad6-895eaa65739f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:47.720898 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 98/192] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:47.721705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436023fa-7734-4425-a566-e6a39ef7cee1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:47.725139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:47.752108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d345af5-6c10-4aa5-93b2-a48ac5a6daa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:47.752831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d345af5-6c10-4aa5-93b2-a48ac5a6daa2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:47.753086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d345af5-6c10-4aa5-93b2-a48ac5a6daa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:47.753275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d345af5-6c10-4aa5-93b2-a48ac5a6daa2 None admin] PATH_INFO: `/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:47.770801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d345af5-6c10-4aa5-93b2-a48ac5a6daa2 None admin] RBAC: Authorizing `identity:get_user(user_id=ce768da38f0f4bf58da2a78ad81ab2a4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:47.772592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d345af5-6c10-4aa5-93b2-a48ac5a6daa2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:47.779028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 95/193] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:47 2022] GET /identity/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:47.779028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d345af5-6c10-4aa5-93b2-a48ac5a6daa2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:47.839270 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436023fa-7734-4425-a566-e6a39ef7cee1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:47.876899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a5258-9a73-4da0-9165-40f18e0b5dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:47.877710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5258-9a73-4da0-9165-40f18e0b5dda None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:47.878148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5258-9a73-4da0-9165-40f18e0b5dda None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:47.878420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5258-9a73-4da0-9165-40f18e0b5dda None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:47.879091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a5258-9a73-4da0-9165-40f18e0b5dda None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:47.879395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a5258-9a73-4da0-9165-40f18e0b5dda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:47.880823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a5258-9a73-4da0-9165-40f18e0b5dda None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:47.883588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d345af5-6c10-4aa5-93b2-a48ac5a6daa2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:47.889184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 99/194] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:49:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:47.889990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a5258-9a73-4da0-9165-40f18e0b5dda None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:47.924394 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80391ca-6f44-42a7-a018-f123eef1475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:47.925043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80391ca-6f44-42a7-a018-f123eef1475c None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:47.925353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80391ca-6f44-42a7-a018-f123eef1475c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:47.925653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80391ca-6f44-42a7-a018-f123eef1475c None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:47.926390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80391ca-6f44-42a7-a018-f123eef1475c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:47.927836 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80391ca-6f44-42a7-a018-f123eef1475c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:47.948823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:47.949619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 96/195] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:49:47 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:49:47.950582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80391ca-6f44-42a7-a018-f123eef1475c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:47.984703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:48.027952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a5258-9a73-4da0-9165-40f18e0b5dda None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:48.068996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e60157-0b02-4714-bbbf-a4140857c98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:48.069621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e60157-0b02-4714-bbbf-a4140857c98a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:48.069892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e60157-0b02-4714-bbbf-a4140857c98a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:48.070166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e60157-0b02-4714-bbbf-a4140857c98a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:48.088704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80391ca-6f44-42a7-a018-f123eef1475c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:48.128440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8ba83-5eb3-4449-8376-0a78ff8c1502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:48.129563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8ba83-5eb3-4449-8376-0a78ff8c1502 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:48.129563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8ba83-5eb3-4449-8376-0a78ff8c1502 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:48.129563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8ba83-5eb3-4449-8376-0a78ff8c1502 None admin] PATH_INFO: `/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:48.130775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8ba83-5eb3-4449-8376-0a78ff8c1502 None admin] RBAC: Authorizing `identity:get_user(user_id=ce768da38f0f4bf58da2a78ad81ab2a4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:48.132247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8ba83-5eb3-4449-8376-0a78ff8c1502 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:48.134056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 97/196] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:47 2022] GET /identity/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:48.137507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8ba83-5eb3-4449-8376-0a78ff8c1502 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:48.162764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e60157-0b02-4714-bbbf-a4140857c98a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:48.167150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e60157-0b02-4714-bbbf-a4140857c98a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:48.236256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:48.258910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:48.270411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8ba83-5eb3-4449-8376-0a78ff8c1502 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:48.309298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 100/197] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:48.310148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e60157-0b02-4714-bbbf-a4140857c98a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:48.316691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0b3f7-0fcd-4dac-bbc2-182f268572d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:48.317632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0b3f7-0fcd-4dac-bbc2-182f268572d0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:48.318130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0b3f7-0fcd-4dac-bbc2-182f268572d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:48.319122 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0b3f7-0fcd-4dac-bbc2-182f268572d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:48.332220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0b3f7-0fcd-4dac-bbc2-182f268572d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-57e0b3f7-0fcd-4dac-bbc2-182f268572d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:48.338104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:48.340230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 98/198] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:48.342152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0b3f7-0fcd-4dac-bbc2-182f268572d0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:48.428225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e60157-0b02-4714-bbbf-a4140857c98a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:48.458929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0b3f7-0fcd-4dac-bbc2-182f268572d0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:48.466910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e52249-ad35-480e-9238-bfe6352495fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:48.467519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e52249-ad35-480e-9238-bfe6352495fd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:48.467796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e52249-ad35-480e-9238-bfe6352495fd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:48.468088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e52249-ad35-480e-9238-bfe6352495fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:48.487610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:48.512027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bfe7e-072e-4b26-b5e8-c44e738f3c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:48.512710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bfe7e-072e-4b26-b5e8-c44e738f3c4b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:48.512984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bfe7e-072e-4b26-b5e8-c44e738f3c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:48.513254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bfe7e-072e-4b26-b5e8-c44e738f3c4b None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:48.514421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bfe7e-072e-4b26-b5e8-c44e738f3c4b None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:48.517399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bfe7e-072e-4b26-b5e8-c44e738f3c4b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:48.519957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 99/199] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:48 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:48.521365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bfe7e-072e-4b26-b5e8-c44e738f3c4b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:48.545319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e52249-ad35-480e-9238-bfe6352495fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:48.547231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e52249-ad35-480e-9238-bfe6352495fd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:48.547969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:48.662116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 101/200] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:48.665866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e52249-ad35-480e-9238-bfe6352495fd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:48.676033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bfe7e-072e-4b26-b5e8-c44e738f3c4b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:48.712846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049b734-5b72-4fcc-8bb0-8f04516b5dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:48.713636 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049b734-5b72-4fcc-8bb0-8f04516b5dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:48.714020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049b734-5b72-4fcc-8bb0-8f04516b5dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:48.714398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049b734-5b72-4fcc-8bb0-8f04516b5dc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:48.715174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049b734-5b72-4fcc-8bb0-8f04516b5dc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:48.715585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049b734-5b72-4fcc-8bb0-8f04516b5dc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:48.717114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049b734-5b72-4fcc-8bb0-8f04516b5dc5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:48.727340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 100/201] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:48.732851 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049b734-5b72-4fcc-8bb0-8f04516b5dc5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:48.745585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:48.819101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e52249-ad35-480e-9238-bfe6352495fd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:48.830944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:48.851713 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049b734-5b72-4fcc-8bb0-8f04516b5dc5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:48.878695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43033c81-14f2-431d-b8e9-b319743d94de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:48.881006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43033c81-14f2-431d-b8e9-b319743d94de None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:48.881845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43033c81-14f2-431d-b8e9-b319743d94de None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:48.882563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43033c81-14f2-431d-b8e9-b319743d94de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:48.884833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43033c81-14f2-431d-b8e9-b319743d94de None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=10e7858e00034a8284453b08e8470748, user.id=ce768da38f0f4bf58da2a78ad81ab2a4) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:48.886133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43033c81-14f2-431d-b8e9-b319743d94de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:48.889875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43033c81-14f2-431d-b8e9-b319743d94de None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:48.905454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 102/202] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:48 2022] GET /identity/v3/role_assignments?user.id=ce768da38f0f4bf58da2a78ad81ab2a4&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:48.905454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43033c81-14f2-431d-b8e9-b319743d94de None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:48.960908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa97d58-1298-47d1-8471-c09fa33a037d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:48.961577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa97d58-1298-47d1-8471-c09fa33a037d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:48.961857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa97d58-1298-47d1-8471-c09fa33a037d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:48.962130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa97d58-1298-47d1-8471-c09fa33a037d None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:48.963497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa97d58-1298-47d1-8471-c09fa33a037d None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:48.964929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa97d58-1298-47d1-8471-c09fa33a037d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:48.966652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 101/203] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:48 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:48.983752 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa97d58-1298-47d1-8471-c09fa33a037d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:49.165736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43033c81-14f2-431d-b8e9-b319743d94de None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:49.186053 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:49.220975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa97d58-1298-47d1-8471-c09fa33a037d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:49.244576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735afdc-f304-4678-bacb-eb763f0b3033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:49.245767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735afdc-f304-4678-bacb-eb763f0b3033 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:49.246273 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735afdc-f304-4678-bacb-eb763f0b3033 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:49.246824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735afdc-f304-4678-bacb-eb763f0b3033 None admin] PATH_INFO: `/v3/users/d70bcc9f75eb448e807921f69100128f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:49.249469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735afdc-f304-4678-bacb-eb763f0b3033 None admin] RBAC: Authorizing `identity:get_user(user_id=d70bcc9f75eb448e807921f69100128f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:49.251238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735afdc-f304-4678-bacb-eb763f0b3033 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:49.253577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 103/204] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:48 2022] GET /identity/v3/users/d70bcc9f75eb448e807921f69100128f => generated 289 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:49.258376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a37d-d5eb-4666-a347-867e2d77ff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:49.259107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a37d-d5eb-4666-a347-867e2d77ff2b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:49.259673 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a37d-d5eb-4666-a347-867e2d77ff2b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:49.260058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a37d-d5eb-4666-a347-867e2d77ff2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:49.260716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735afdc-f304-4678-bacb-eb763f0b3033 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:49.264066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4a37d-d5eb-4666-a347-867e2d77ff2b None admin] RBAC: Adding query filter params (role.id=fc1186ecd0f74b2fad8155d528f268f9, scope.project.id=10e7858e00034a8284453b08e8470748, user.id=ce768da38f0f4bf58da2a78ad81ab2a4) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:49.264066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4a37d-d5eb-4666-a347-867e2d77ff2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:49.264430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4a37d-d5eb-4666-a347-867e2d77ff2b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:49.270868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 102/205] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:48 2022] GET /identity/v3/role_assignments?user.id=ce768da38f0f4bf58da2a78ad81ab2a4&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=fc1186ecd0f74b2fad8155d528f268f9 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:49.434311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735afdc-f304-4678-bacb-eb763f0b3033 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:49.476358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4857814-1471-4f5c-ac5b-1b813d93cbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:49.476900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4857814-1471-4f5c-ac5b-1b813d93cbcd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:49.477187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4857814-1471-4f5c-ac5b-1b813d93cbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:49.477457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4857814-1471-4f5c-ac5b-1b813d93cbcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:49.550037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:49.578849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4857814-1471-4f5c-ac5b-1b813d93cbcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:49.580992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4857814-1471-4f5c-ac5b-1b813d93cbcd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:49.711992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 104/206] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:49.721679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a37d-d5eb-4666-a347-867e2d77ff2b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:49.752539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:49.880641 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a37d-d5eb-4666-a347-867e2d77ff2b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:49.931662 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d19a5-c7f7-40d3-9032-7adb028d481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:49.932347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d19a5-c7f7-40d3-9032-7adb028d481b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:49.932635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d19a5-c7f7-40d3-9032-7adb028d481b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:49.932915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d19a5-c7f7-40d3-9032-7adb028d481b None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:49.934069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d19a5-c7f7-40d3-9032-7adb028d481b None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:49.935534 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d19a5-c7f7-40d3-9032-7adb028d481b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:49.937424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 103/207] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:49 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:49.945160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4857814-1471-4f5c-ac5b-1b813d93cbcd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:50.064753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:50.088878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4857814-1471-4f5c-ac5b-1b813d93cbcd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:50.134618 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6dace4-c524-47a7-80d4-087c468825e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:50.135217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dace4-c524-47a7-80d4-087c468825e1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:50.135494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dace4-c524-47a7-80d4-087c468825e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:50.135763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dace4-c524-47a7-80d4-087c468825e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:50.136530 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6dace4-c524-47a7-80d4-087c468825e1 None admin] RBAC: Adding query filter params (role.id=7fcd054cc7c549b18e4ac97d4a405e8c, scope.project.id=2ec79012680c43e3bfc414986fbb0b79, user.id=d70bcc9f75eb448e807921f69100128f) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:50.136836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6dace4-c524-47a7-80d4-087c468825e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:50.138296 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6dace4-c524-47a7-80d4-087c468825e1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:50.144411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 105/208] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:49 2022] GET /identity/v3/role_assignments?user.id=d70bcc9f75eb448e807921f69100128f&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=7fcd054cc7c549b18e4ac97d4a405e8c => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:50.208375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b2623-5309-4119-bddb-868991051523 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:50.209068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b2623-5309-4119-bddb-868991051523 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:50.209668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b2623-5309-4119-bddb-868991051523 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:50.211553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 104/209] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:50.222901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e86937-8bf3-4103-8666-db6bc983464f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:50.223245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e86937-8bf3-4103-8666-db6bc983464f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:50.223460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e86937-8bf3-4103-8666-db6bc983464f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:50.249429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a8e86937-8bf3-4103-8666-db6bc983464f None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:50.250681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a8e86937-8bf3-4103-8666-db6bc983464f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:50.340874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 106/210] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:50.434473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f58854-3d30-45ec-a136-5b7de84043b6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:50.434999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f58854-3d30-45ec-a136-5b7de84043b6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:50.434999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f58854-3d30-45ec-a136-5b7de84043b6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:50.436674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 105/211] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:50.441990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513aeeb-fce5-463c-84cf-9d7204dc8fb5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:50.442447 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513aeeb-fce5-463c-84cf-9d7204dc8fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:50.442883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513aeeb-fce5-463c-84cf-9d7204dc8fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:50.462743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7513aeeb-fce5-463c-84cf-9d7204dc8fb5 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:50.464247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7513aeeb-fce5-463c-84cf-9d7204dc8fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:50.544948 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 107/212] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:50.628556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f58854-3d30-45ec-a136-5b7de84043b6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:50.760468 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:50.784533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f58854-3d30-45ec-a136-5b7de84043b6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:50.829266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a295762-7119-4a49-b528-5c65013d9d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:50.829991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a295762-7119-4a49-b528-5c65013d9d3b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:50.830274 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a295762-7119-4a49-b528-5c65013d9d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:50.830549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a295762-7119-4a49-b528-5c65013d9d3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:50.844616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a295762-7119-4a49-b528-5c65013d9d3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0a295762-7119-4a49-b528-5c65013d9d3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:50.851276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:50.852217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:50.853010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 106/213] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:50.862680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513aeeb-fce5-463c-84cf-9d7204dc8fb5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:50.870569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a295762-7119-4a49-b528-5c65013d9d3b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:50.874210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:51.017866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513aeeb-fce5-463c-84cf-9d7204dc8fb5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:51.037005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a295762-7119-4a49-b528-5c65013d9d3b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:51.069935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99701a24-4400-46a4-a3d6-02d674755875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:51.070925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99701a24-4400-46a4-a3d6-02d674755875 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:51.071401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99701a24-4400-46a4-a3d6-02d674755875 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:51.071878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99701a24-4400-46a4-a3d6-02d674755875 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:51.072751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99701a24-4400-46a4-a3d6-02d674755875 None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:51.073242 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99701a24-4400-46a4-a3d6-02d674755875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:51.074511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7136b31-1621-4148-ba42-2005e982fe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:51.074936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99701a24-4400-46a4-a3d6-02d674755875 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:51.075386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7136b31-1621-4148-ba42-2005e982fe19 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:51.075766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7136b31-1621-4148-ba42-2005e982fe19 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:51.076222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7136b31-1621-4148-ba42-2005e982fe19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:51.077455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7136b31-1621-4148-ba42-2005e982fe19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:51.079004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7136b31-1621-4148-ba42-2005e982fe19 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:51.084330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 107/214] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:51.086831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7136b31-1621-4148-ba42-2005e982fe19 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:51.097788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 108/215] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:51.098914 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99701a24-4400-46a4-a3d6-02d674755875 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:51.103707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:51.225644 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:51.241144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7136b31-1621-4148-ba42-2005e982fe19 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:51.290832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0036b5-9e9f-4fbb-a04a-814bef6b6855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:51.292716 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0036b5-9e9f-4fbb-a04a-814bef6b6855 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:51.293862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0036b5-9e9f-4fbb-a04a-814bef6b6855 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:51.294334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0036b5-9e9f-4fbb-a04a-814bef6b6855 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:51.295186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0036b5-9e9f-4fbb-a04a-814bef6b6855 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:51.297051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0036b5-9e9f-4fbb-a04a-814bef6b6855 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:51.304555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99701a24-4400-46a4-a3d6-02d674755875 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:51.332688 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 108/216] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:49:51 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:49:51.343139 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf02dc-45f5-49bc-a69f-ee69949f50c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:51.343782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf02dc-45f5-49bc-a69f-ee69949f50c3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:51.344073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf02dc-45f5-49bc-a69f-ee69949f50c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:51.344373 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf02dc-45f5-49bc-a69f-ee69949f50c3 None admin] PATH_INFO: `/v3/users/ac1b33d6e6434973bdb8f3883c988035` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:51.360379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf02dc-45f5-49bc-a69f-ee69949f50c3 None admin] RBAC: Authorizing `identity:get_user(user_id=ac1b33d6e6434973bdb8f3883c988035)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:51.362409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf02dc-45f5-49bc-a69f-ee69949f50c3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:51.364386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 109/217] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:51 2022] GET /identity/v3/users/ac1b33d6e6434973bdb8f3883c988035 => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:51.370009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0036b5-9e9f-4fbb-a04a-814bef6b6855 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:51.414935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4964b-60c7-4ce8-8e0d-5f9ed7b9dd69 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:51.415244 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4964b-60c7-4ce8-8e0d-5f9ed7b9dd69 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:51.415493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4964b-60c7-4ce8-8e0d-5f9ed7b9dd69 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:51.417261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 110/218] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:51.422839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ffaf0-11d9-4df5-afad-e1ddba04f27a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:51.423003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ffaf0-11d9-4df5-afad-e1ddba04f27a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:51.423222 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ffaf0-11d9-4df5-afad-e1ddba04f27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:51.442672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6b9ffaf0-11d9-4df5-afad-e1ddba04f27a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:51.443888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6b9ffaf0-11d9-4df5-afad-e1ddba04f27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:51.449965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:51.504527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0036b5-9e9f-4fbb-a04a-814bef6b6855 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:51.543785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 111/219] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:51.544752 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120ba2-6503-43f0-aaaa-7a9ff989205d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:51.545039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120ba2-6503-43f0-aaaa-7a9ff989205d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:51.545309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120ba2-6503-43f0-aaaa-7a9ff989205d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:51.548156 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 112/220] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:51.548287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b55b0c-c187-4544-b2cf-4aca1db66f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:51.548287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b55b0c-c187-4544-b2cf-4aca1db66f5d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:51.548287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b55b0c-c187-4544-b2cf-4aca1db66f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:51.548287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b55b0c-c187-4544-b2cf-4aca1db66f5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:51.552713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a2979-8156-4a5b-a8ed-682594ecc0e0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:51.553194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a2979-8156-4a5b-a8ed-682594ecc0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:51.553638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a2979-8156-4a5b-a8ed-682594ecc0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:51.573384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e25a2979-8156-4a5b-a8ed-682594ecc0e0 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:51.574844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e25a2979-8156-4a5b-a8ed-682594ecc0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:51.635400 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:51.643919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b55b0c-c187-4544-b2cf-4aca1db66f5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:51.645435 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b55b0c-c187-4544-b2cf-4aca1db66f5d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:51.657695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 113/221] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:51.715631 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:51.759747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 109/222] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:51.765584 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a2979-8156-4a5b-a8ed-682594ecc0e0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:51.842480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b55b0c-c187-4544-b2cf-4aca1db66f5d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:51.892844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a2979-8156-4a5b-a8ed-682594ecc0e0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:51.934818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af8fca-bf4f-4414-b2dd-bbd614293631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:51.935514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af8fca-bf4f-4414-b2dd-bbd614293631 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:51.935852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af8fca-bf4f-4414-b2dd-bbd614293631 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:51.936219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af8fca-bf4f-4414-b2dd-bbd614293631 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:51.937357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af8fca-bf4f-4414-b2dd-bbd614293631 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:51.938813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af8fca-bf4f-4414-b2dd-bbd614293631 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:51.940870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 114/223] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:51 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:51.949525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af8fca-bf4f-4414-b2dd-bbd614293631 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:51.987129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b55b0c-c187-4544-b2cf-4aca1db66f5d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:52.021258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:52.032012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dece8b-5df3-4989-8ac4-bdb12cd91e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:52.032771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dece8b-5df3-4989-8ac4-bdb12cd91e2f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:52.033618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dece8b-5df3-4989-8ac4-bdb12cd91e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:52.033894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dece8b-5df3-4989-8ac4-bdb12cd91e2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:52.046814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dece8b-5df3-4989-8ac4-bdb12cd91e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-62dece8b-5df3-4989-8ac4-bdb12cd91e2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:52.053550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:52.054432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:52.055217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 110/224] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:49:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:49:52.060742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dece8b-5df3-4989-8ac4-bdb12cd91e2f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:52.090106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af8fca-bf4f-4414-b2dd-bbd614293631 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:52.092571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:52.131367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8969ac-0525-499d-bf89-68bee410669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:52.131980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8969ac-0525-499d-bf89-68bee410669f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:52.132380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8969ac-0525-499d-bf89-68bee410669f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:52.132656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8969ac-0525-499d-bf89-68bee410669f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:52.133391 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8969ac-0525-499d-bf89-68bee410669f None admin] RBAC: Adding query filter params (role.id=7ab2687826544b8e80c709d6061162d4, scope.project.id=10e7858e00034a8284453b08e8470748, user.id=ac1b33d6e6434973bdb8f3883c988035) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:52.133703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8969ac-0525-499d-bf89-68bee410669f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:52.135341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8969ac-0525-499d-bf89-68bee410669f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:52.141570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 115/225] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:51 2022] GET /identity/v3/role_assignments?user.id=ac1b33d6e6434973bdb8f3883c988035&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=7ab2687826544b8e80c709d6061162d4 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:52.143090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8969ac-0525-499d-bf89-68bee410669f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:52.196767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dece8b-5df3-4989-8ac4-bdb12cd91e2f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:52.284878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8969ac-0525-499d-bf89-68bee410669f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:52.288792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca988afa-2a14-4727-ba12-977b295db746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:52.293230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca988afa-2a14-4727-ba12-977b295db746 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:52.293230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca988afa-2a14-4727-ba12-977b295db746 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:52.293230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca988afa-2a14-4727-ba12-977b295db746 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:52.304453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:52.321665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca988afa-2a14-4727-ba12-977b295db746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ca988afa-2a14-4727-ba12-977b295db746 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:52.327923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:52.329076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:52.330109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 111/226] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:52.336785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca988afa-2a14-4727-ba12-977b295db746 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:52.339386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c6076-5ebf-4c27-9161-4fff3769c068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:52.340235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c6076-5ebf-4c27-9161-4fff3769c068 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:52.340906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c6076-5ebf-4c27-9161-4fff3769c068 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:52.341325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c6076-5ebf-4c27-9161-4fff3769c068 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:52.343135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c6076-5ebf-4c27-9161-4fff3769c068 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:52.343591 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c6076-5ebf-4c27-9161-4fff3769c068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:52.345173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c6076-5ebf-4c27-9161-4fff3769c068 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:52.362932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 116/227] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:49:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:52.368661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c6076-5ebf-4c27-9161-4fff3769c068 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:52.452418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:52.469918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca988afa-2a14-4727-ba12-977b295db746 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:52.508725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ff17b-8f66-4c01-b075-11a48b49d86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:52.509714 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff17b-8f66-4c01-b075-11a48b49d86d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:52.510190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff17b-8f66-4c01-b075-11a48b49d86d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:52.510657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff17b-8f66-4c01-b075-11a48b49d86d None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:52.511513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ff17b-8f66-4c01-b075-11a48b49d86d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:52.512010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ff17b-8f66-4c01-b075-11a48b49d86d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:52.514126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ff17b-8f66-4c01-b075-11a48b49d86d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:52.523893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 112/228] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:52.529511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ff17b-8f66-4c01-b075-11a48b49d86d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:52.626059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c6076-5ebf-4c27-9161-4fff3769c068 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:52.643302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ff17b-8f66-4c01-b075-11a48b49d86d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:52.681474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a65f6-c6fd-46c2-a15b-2c91c3a214e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:52.682477 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a65f6-c6fd-46c2-a15b-2c91c3a214e5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:52.683051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a65f6-c6fd-46c2-a15b-2c91c3a214e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:52.683542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a65f6-c6fd-46c2-a15b-2c91c3a214e5 None admin] PATH_INFO: `/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:52.685073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a65f6-c6fd-46c2-a15b-2c91c3a214e5 None admin] RBAC: Authorizing `identity:get_user(user_id=ce768da38f0f4bf58da2a78ad81ab2a4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:52.686822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a65f6-c6fd-46c2-a15b-2c91c3a214e5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:52.689149 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 113/229] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:52 2022] GET /identity/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:52.697300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a65f6-c6fd-46c2-a15b-2c91c3a214e5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:52.718004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d537e-d18f-405e-8d23-cf7557f7d9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:52.719082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d537e-d18f-405e-8d23-cf7557f7d9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:52.719571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d537e-d18f-405e-8d23-cf7557f7d9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:52.720035 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d537e-d18f-405e-8d23-cf7557f7d9e3 None admin] PATH_INFO: `/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:52.723226 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d537e-d18f-405e-8d23-cf7557f7d9e3 None admin] RBAC: Authorizing `identity:get_user(user_id=ce768da38f0f4bf58da2a78ad81ab2a4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:52.726446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d537e-d18f-405e-8d23-cf7557f7d9e3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:52.730150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 117/230] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:52 2022] GET /identity/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4 => generated 283 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:52.731773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:52.735490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d537e-d18f-405e-8d23-cf7557f7d9e3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:52.850508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:52.860638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a65f6-c6fd-46c2-a15b-2c91c3a214e5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:52.898502 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79080f-d64a-4735-9adb-bfe192da7cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:52.899237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79080f-d64a-4735-9adb-bfe192da7cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:52.899739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79080f-d64a-4735-9adb-bfe192da7cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:52.900147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79080f-d64a-4735-9adb-bfe192da7cd9 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:52.901638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f79080f-d64a-4735-9adb-bfe192da7cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:52.903341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f79080f-d64a-4735-9adb-bfe192da7cd9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:52.905610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 114/231] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:52 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:52.911657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79080f-d64a-4735-9adb-bfe192da7cd9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:52.989761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d537e-d18f-405e-8d23-cf7557f7d9e3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:53.060018 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61255004-3edb-4245-8585-5593c0dce027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:53.060685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61255004-3edb-4245-8585-5593c0dce027 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:53.060962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61255004-3edb-4245-8585-5593c0dce027 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:53.061235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61255004-3edb-4245-8585-5593c0dce027 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:53.064085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61255004-3edb-4245-8585-5593c0dce027 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:53.066123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61255004-3edb-4245-8585-5593c0dce027 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:53.071734 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 118/232] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:52 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:53.076174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:53.077029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61255004-3edb-4245-8585-5593c0dce027 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:53.097647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79080f-d64a-4735-9adb-bfe192da7cd9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:53.141409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da83ed-53cc-402a-aba4-a1a2450eda58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:53.142075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da83ed-53cc-402a-aba4-a1a2450eda58 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:53.142355 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da83ed-53cc-402a-aba4-a1a2450eda58 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:53.143195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da83ed-53cc-402a-aba4-a1a2450eda58 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748/users/ce768da38f0f4bf58da2a78ad81ab2a4/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:53.146319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da83ed-53cc-402a-aba4-a1a2450eda58 None admin] RBAC: Authorizing `identity:create_grant(project_id=10e7858e00034a8284453b08e8470748, user_id=ce768da38f0f4bf58da2a78ad81ab2a4, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:53.147765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da83ed-53cc-402a-aba4-a1a2450eda58 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:53.174655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 115/233] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:49:52 2022] PUT /identity/v3/projects/10e7858e00034a8284453b08e8470748/users/ce768da38f0f4bf58da2a78ad81ab2a4/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:49:53.265363 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61255004-3edb-4245-8585-5593c0dce027 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:53.289473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:53.319555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157abb43-2f2a-4fdd-8a73-1d2b2568e47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:53.320256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157abb43-2f2a-4fdd-8a73-1d2b2568e47b None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:53.320540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157abb43-2f2a-4fdd-8a73-1d2b2568e47b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:53.320817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157abb43-2f2a-4fdd-8a73-1d2b2568e47b None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748/users/ce768da38f0f4bf58da2a78ad81ab2a4/roles/fc1186ecd0f74b2fad8155d528f268f9` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:53.329240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157abb43-2f2a-4fdd-8a73-1d2b2568e47b None admin] RBAC: Authorizing `identity:create_grant(project_id=10e7858e00034a8284453b08e8470748, user_id=ce768da38f0f4bf58da2a78ad81ab2a4, role_id=fc1186ecd0f74b2fad8155d528f268f9)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:53.330791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157abb43-2f2a-4fdd-8a73-1d2b2568e47b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:53.347682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 119/234] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:49:53 2022] PUT /identity/v3/projects/10e7858e00034a8284453b08e8470748/users/ce768da38f0f4bf58da2a78ad81ab2a4/roles/fc1186ecd0f74b2fad8155d528f268f9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:49:53.799163 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff92eb-2d79-42d0-8f58-abafc8770d29 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:53.800133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff92eb-2d79-42d0-8f58-abafc8770d29 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:53.800673 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff92eb-2d79-42d0-8f58-abafc8770d29 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:53.802219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 116/235] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:53.808133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe4128-7841-4148-afd0-d9adc2786e29 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:53.808824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe4128-7841-4148-afd0-d9adc2786e29 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:53.809314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe4128-7841-4148-afd0-d9adc2786e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:53.827814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f12c-b369-4c8c-8a1e-dfe8cf611b90 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:53.828452 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f12c-b369-4c8c-8a1e-dfe8cf611b90 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:53.828918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f12c-b369-4c8c-8a1e-dfe8cf611b90 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:53.831119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 117/236] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:49:53.831471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fdfe4128-7841-4148-afd0-d9adc2786e29 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:53.837697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fdfe4128-7841-4148-afd0-d9adc2786e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:53.849444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f2b34-b8e3-46bb-ace2-784c03c24dc3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:53.850040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f2b34-b8e3-46bb-ace2-784c03c24dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:53.850531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f2b34-b8e3-46bb-ace2-784c03c24dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:53.882884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c39f2b34-b8e3-46bb-ace2-784c03c24dc3 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:53.884472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c39f2b34-b8e3-46bb-ace2-784c03c24dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:53.913494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 120/237] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:54.010240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 118/238] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:54.336157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe4128-7841-4148-afd0-d9adc2786e29 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:54.363857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f2b34-b8e3-46bb-ace2-784c03c24dc3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:54.488988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:54.512621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f2b34-b8e3-46bb-ace2-784c03c24dc3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:54.513839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe4128-7841-4148-afd0-d9adc2786e29 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:54.518691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:54.561125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dfb281-2680-461a-a3af-765db0e2ebd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:54.561861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfb281-2680-461a-a3af-765db0e2ebd2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:54.562143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfb281-2680-461a-a3af-765db0e2ebd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:54.562415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfb281-2680-461a-a3af-765db0e2ebd2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:54.562796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d3e62-9c00-473d-99ab-adb735860cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:54.563523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d3e62-9c00-473d-99ab-adb735860cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:54.563899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d3e62-9c00-473d-99ab-adb735860cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:54.565053 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d3e62-9c00-473d-99ab-adb735860cb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:54.566685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d3e62-9c00-473d-99ab-adb735860cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:54.568587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d3e62-9c00-473d-99ab-adb735860cb7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:54.570548 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 121/239] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:54.572085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0cd94-123e-4b76-ab91-feb06fdbcd5c None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:54.572472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0cd94-123e-4b76-ab91-feb06fdbcd5c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:54.572850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0cd94-123e-4b76-ab91-feb06fdbcd5c None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:54.573760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dfb281-2680-461a-a3af-765db0e2ebd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:54.578455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 122/240] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-14dfb281-2680-461a-a3af-765db0e2ebd2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:54.578826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:54.579704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:49:54.581158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:54.581761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0cd94-123e-4b76-ab91-feb06fdbcd5c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:54.581827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 119/241] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:54.581898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df214481-110f-4e46-b3ad-7d4696be0ab9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:54.586591 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df214481-110f-4e46-b3ad-7d4696be0ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:54.586591 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df214481-110f-4e46-b3ad-7d4696be0ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:54.602211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-df214481-110f-4e46-b3ad-7d4696be0ab9 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:54.603507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-df214481-110f-4e46-b3ad-7d4696be0ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:54.681463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 120/242] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:54.682430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df214481-110f-4e46-b3ad-7d4696be0ab9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:54.729757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0cd94-123e-4b76-ab91-feb06fdbcd5c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:54.785866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f08b5-396e-4d8e-972c-5f560c1ba938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:54.785866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f08b5-396e-4d8e-972c-5f560c1ba938 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:54.785866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f08b5-396e-4d8e-972c-5f560c1ba938 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:54.785866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f08b5-396e-4d8e-972c-5f560c1ba938 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:54.786240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f08b5-396e-4d8e-972c-5f560c1ba938 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:54.786970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f08b5-396e-4d8e-972c-5f560c1ba938 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:54.805587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:54.822275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 123/243] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:49:54 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:49:54.867954 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df214481-110f-4e46-b3ad-7d4696be0ab9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:54.908798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b14e5-2279-4904-9bb1-0f9a87fc2a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:54.909429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b14e5-2279-4904-9bb1-0f9a87fc2a9c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:54.909709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b14e5-2279-4904-9bb1-0f9a87fc2a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:54.909985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b14e5-2279-4904-9bb1-0f9a87fc2a9c None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:54.910681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b14e5-2279-4904-9bb1-0f9a87fc2a9c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:54.910986 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b14e5-2279-4904-9bb1-0f9a87fc2a9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:54.912467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b14e5-2279-4904-9bb1-0f9a87fc2a9c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:54.922926 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 121/244] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:54.933430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f08b5-396e-4d8e-972c-5f560c1ba938 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:54.975485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:55.049715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b14e5-2279-4904-9bb1-0f9a87fc2a9c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:55.071247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f08b5-396e-4d8e-972c-5f560c1ba938 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:55.110468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c651b0-78a2-41f0-966a-13daaa8e07f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:55.111232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c651b0-78a2-41f0-966a-13daaa8e07f9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:55.112071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c651b0-78a2-41f0-966a-13daaa8e07f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:55.112371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c651b0-78a2-41f0-966a-13daaa8e07f9 None admin] PATH_INFO: `/v3/users/c1f0fd904ad74d8ba3c262518f23a93c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:55.131836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c651b0-78a2-41f0-966a-13daaa8e07f9 None admin] RBAC: Authorizing `identity:get_user(user_id=c1f0fd904ad74d8ba3c262518f23a93c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:55.133290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c651b0-78a2-41f0-966a-13daaa8e07f9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:55.136177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 124/245] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:54 2022] GET /identity/v3/users/c1f0fd904ad74d8ba3c262518f23a93c => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:55.141885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c651b0-78a2-41f0-966a-13daaa8e07f9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:55.177090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b14e5-2279-4904-9bb1-0f9a87fc2a9c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:55.216457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:55.239537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905666f4-332d-4abd-bb62-555d377c92f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:55.240346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905666f4-332d-4abd-bb62-555d377c92f5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:55.240897 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905666f4-332d-4abd-bb62-555d377c92f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:55.241170 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905666f4-332d-4abd-bb62-555d377c92f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:55.257046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905666f4-332d-4abd-bb62-555d377c92f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-905666f4-332d-4abd-bb62-555d377c92f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:55.264197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:55.265072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:55.265947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 122/246] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:55.271780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905666f4-332d-4abd-bb62-555d377c92f5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:55.281970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:55.308515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c651b0-78a2-41f0-966a-13daaa8e07f9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:55.351890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b40ad-9541-45d4-b81b-c5bc8918e104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:55.352684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b40ad-9541-45d4-b81b-c5bc8918e104 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:55.353031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b40ad-9541-45d4-b81b-c5bc8918e104 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:55.353370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b40ad-9541-45d4-b81b-c5bc8918e104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:55.419039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905666f4-332d-4abd-bb62-555d377c92f5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:55.441222 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b40ad-9541-45d4-b81b-c5bc8918e104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:55.443270 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b40ad-9541-45d4-b81b-c5bc8918e104 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:55.465714 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a5a6b-c283-4641-a14d-be13d380f0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:55.466321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a5a6b-c283-4641-a14d-be13d380f0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:55.466599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a5a6b-c283-4641-a14d-be13d380f0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:55.466870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a5a6b-c283-4641-a14d-be13d380f0d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:55.467541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a5a6b-c283-4641-a14d-be13d380f0d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:55.467837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a5a6b-c283-4641-a14d-be13d380f0d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:55.469343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a5a6b-c283-4641-a14d-be13d380f0d4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:55.478738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 123/247] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:55.489593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a5a6b-c283-4641-a14d-be13d380f0d4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:55.546349 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:55.551975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:55.564027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 125/248] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:49:55.570652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b40ad-9541-45d4-b81b-c5bc8918e104 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:55.620206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a5a6b-c283-4641-a14d-be13d380f0d4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:55.660338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207fff5-4fae-47bb-b04e-0389304ce0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:55.661011 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207fff5-4fae-47bb-b04e-0389304ce0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:55.661294 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207fff5-4fae-47bb-b04e-0389304ce0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:55.661569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207fff5-4fae-47bb-b04e-0389304ce0b2 None admin] PATH_INFO: `/v3/users/ac1b33d6e6434973bdb8f3883c988035` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:55.663536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207fff5-4fae-47bb-b04e-0389304ce0b2 None admin] RBAC: Authorizing `identity:get_user(user_id=ac1b33d6e6434973bdb8f3883c988035)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:55.665023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207fff5-4fae-47bb-b04e-0389304ce0b2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:55.667225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 124/249] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:55 2022] GET /identity/v3/users/ac1b33d6e6434973bdb8f3883c988035 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:55.672696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fa108-3f44-4942-85e9-906dbd639640 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:55.672915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fa108-3f44-4942-85e9-906dbd639640 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:55.673179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fa108-3f44-4942-85e9-906dbd639640 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:55.674968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 125/250] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:55.676333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fa108-3f44-4942-85e9-906dbd639640 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:55.698826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b40ad-9541-45d4-b81b-c5bc8918e104 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:55.744300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e360d-dec2-4187-b4ef-003cf5621754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:55.744945 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e360d-dec2-4187-b4ef-003cf5621754 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:55.745227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e360d-dec2-4187-b4ef-003cf5621754 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:55.745502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e360d-dec2-4187-b4ef-003cf5621754 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:55.747012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e360d-dec2-4187-b4ef-003cf5621754 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:55.748532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e360d-dec2-4187-b4ef-003cf5621754 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:55.750760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 126/251] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:55 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:55.751907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e64b4-b42e-4ab1-8735-085ee6b7c141 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:55.752346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e64b4-b42e-4ab1-8735-085ee6b7c141 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:55.752626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e64b4-b42e-4ab1-8735-085ee6b7c141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:55.777198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b30e64b4-b42e-4ab1-8735-085ee6b7c141 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:55.778613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b30e64b4-b42e-4ab1-8735-085ee6b7c141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:55.795691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fa108-3f44-4942-85e9-906dbd639640 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:55.810669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:55.832802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19dfeb-3f13-45ee-a585-a15df68b8317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:55.833435 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19dfeb-3f13-45ee-a585-a15df68b8317 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:55.833713 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19dfeb-3f13-45ee-a585-a15df68b8317 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:55.833989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19dfeb-3f13-45ee-a585-a15df68b8317 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:55.835157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19dfeb-3f13-45ee-a585-a15df68b8317 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:55.836882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19dfeb-3f13-45ee-a585-a15df68b8317 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:55.838538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 126/252] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:55 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:55.839463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19dfeb-3f13-45ee-a585-a15df68b8317 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:55.860536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:55.897320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 127/253] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:55.898257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e64b4-b42e-4ab1-8735-085ee6b7c141 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:55.952296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19dfeb-3f13-45ee-a585-a15df68b8317 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:56.012333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfecfb-9f6c-4c90-a707-d572da48c287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:56.013010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfecfb-9f6c-4c90-a707-d572da48c287 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:56.013293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfecfb-9f6c-4c90-a707-d572da48c287 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:56.013567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfecfb-9f6c-4c90-a707-d572da48c287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:56.014302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfecfb-9f6c-4c90-a707-d572da48c287 None admin] RBAC: Adding query filter params (role.id=7fcd054cc7c549b18e4ac97d4a405e8c, scope.project.id=10e7858e00034a8284453b08e8470748, user.id=c1f0fd904ad74d8ba3c262518f23a93c) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:56.014614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfecfb-9f6c-4c90-a707-d572da48c287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:56.016094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfecfb-9f6c-4c90-a707-d572da48c287 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:56.022732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 127/254] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:55 2022] GET /identity/v3/role_assignments?user.id=c1f0fd904ad74d8ba3c262518f23a93c&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=7fcd054cc7c549b18e4ac97d4a405e8c => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:56.024085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d877e3a-3c36-44d4-8c6b-daccbf2b7a46 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:56.024364 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d877e3a-3c36-44d4-8c6b-daccbf2b7a46 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:56.024643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d877e3a-3c36-44d4-8c6b-daccbf2b7a46 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:56.026000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 128/255] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:56.034644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb739868-8914-478c-af21-ef05beb00230 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:56.035113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb739868-8914-478c-af21-ef05beb00230 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:56.035581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb739868-8914-478c-af21-ef05beb00230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:56.057891 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-eb739868-8914-478c-af21-ef05beb00230 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:56.059355 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eb739868-8914-478c-af21-ef05beb00230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:56.120484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:56.126411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e64b4-b42e-4ab1-8735-085ee6b7c141 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:56.144900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 129/256] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:56.215238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8222b-a7c9-46ff-bcc7-7e62f57e3aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:56.215874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8222b-a7c9-46ff-bcc7-7e62f57e3aff None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:56.216177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8222b-a7c9-46ff-bcc7-7e62f57e3aff None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:56.216478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8222b-a7c9-46ff-bcc7-7e62f57e3aff None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748/users/ac1b33d6e6434973bdb8f3883c988035/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:56.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8222b-a7c9-46ff-bcc7-7e62f57e3aff None admin] RBAC: Authorizing `identity:create_grant(project_id=10e7858e00034a8284453b08e8470748, user_id=ac1b33d6e6434973bdb8f3883c988035, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:56.223030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8222b-a7c9-46ff-bcc7-7e62f57e3aff None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:56.247309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 128/257] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:49:55 2022] PUT /identity/v3/projects/10e7858e00034a8284453b08e8470748/users/ac1b33d6e6434973bdb8f3883c988035/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:49:56.273491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb739868-8914-478c-af21-ef05beb00230 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:56.403119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb739868-8914-478c-af21-ef05beb00230 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:56.444618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85d086-7e25-4d0b-aeb9-9cfb55c00018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:56.445254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85d086-7e25-4d0b-aeb9-9cfb55c00018 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:56.445532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85d086-7e25-4d0b-aeb9-9cfb55c00018 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:56.445813 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85d086-7e25-4d0b-aeb9-9cfb55c00018 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:56.456891 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85d086-7e25-4d0b-aeb9-9cfb55c00018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-fd85d086-7e25-4d0b-aeb9-9cfb55c00018 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:56.461700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:56.462581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:56.463361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 130/258] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:49:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:56.468885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8222b-a7c9-46ff-bcc7-7e62f57e3aff None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:56.519659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:56.578724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8222b-a7c9-46ff-bcc7-7e62f57e3aff None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:56.617075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8676df-53d7-42be-9797-21cb1e2ca2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:56.617989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8676df-53d7-42be-9797-21cb1e2ca2bb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:56.618486 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8676df-53d7-42be-9797-21cb1e2ca2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:56.618934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8676df-53d7-42be-9797-21cb1e2ca2bb None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:56.619789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8676df-53d7-42be-9797-21cb1e2ca2bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:56.620366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8676df-53d7-42be-9797-21cb1e2ca2bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:56.622196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8676df-53d7-42be-9797-21cb1e2ca2bb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:56.631866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 129/259] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:56.637989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85d086-7e25-4d0b-aeb9-9cfb55c00018 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:56.671088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8676df-53d7-42be-9797-21cb1e2ca2bb None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:56.691090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:56.787905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8676df-53d7-42be-9797-21cb1e2ca2bb None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:56.816478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:56.832565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2ccc3-89c6-4aaf-ac8c-4ffad85c5391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:56.833230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2ccc3-89c6-4aaf-ac8c-4ffad85c5391 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:56.833516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2ccc3-89c6-4aaf-ac8c-4ffad85c5391 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:56.833796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2ccc3-89c6-4aaf-ac8c-4ffad85c5391 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:56.854602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2ccc3-89c6-4aaf-ac8c-4ffad85c5391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-64c2ccc3-89c6-4aaf-ac8c-4ffad85c5391 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:56.860455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:56.861442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:56.862364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 130/260] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:49:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:56.868166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2ccc3-89c6-4aaf-ac8c-4ffad85c5391 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:56.894707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85d086-7e25-4d0b-aeb9-9cfb55c00018 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:56.978414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a1797-895f-4062-8228-898064cbe650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:56.979064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a1797-895f-4062-8228-898064cbe650 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:56.979342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a1797-895f-4062-8228-898064cbe650 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:56.979618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a1797-895f-4062-8228-898064cbe650 None admin] PATH_INFO: `/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:56.986390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a1797-895f-4062-8228-898064cbe650 None admin] RBAC: Authorizing `identity:get_user(user_id=ce768da38f0f4bf58da2a78ad81ab2a4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:56.987971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a1797-895f-4062-8228-898064cbe650 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:56.989644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 131/261] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:56 2022] GET /identity/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4 => generated 283 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:56.996140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a1797-895f-4062-8228-898064cbe650 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:57.073603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2ccc3-89c6-4aaf-ac8c-4ffad85c5391 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:57.110427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e6cb5-1f34-4809-bb42-d7eb0ffdbc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:57.111036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e6cb5-1f34-4809-bb42-d7eb0ffdbc3e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:57.111373 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e6cb5-1f34-4809-bb42-d7eb0ffdbc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:57.111654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e6cb5-1f34-4809-bb42-d7eb0ffdbc3e None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:57.112943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e6cb5-1f34-4809-bb42-d7eb0ffdbc3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:57.113247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e6cb5-1f34-4809-bb42-d7eb0ffdbc3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:57.114639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e6cb5-1f34-4809-bb42-d7eb0ffdbc3e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:57.124026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 131/262] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:49:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:57.130883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e6cb5-1f34-4809-bb42-d7eb0ffdbc3e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:57.188415 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:57.198089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a1797-895f-4062-8228-898064cbe650 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:57.206727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:57.275102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451058dc-8cb8-4498-bf43-a579dc3a85db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:57.275715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451058dc-8cb8-4498-bf43-a579dc3a85db None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:57.275997 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451058dc-8cb8-4498-bf43-a579dc3a85db None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:57.276342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451058dc-8cb8-4498-bf43-a579dc3a85db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:57.324504 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e6cb5-1f34-4809-bb42-d7eb0ffdbc3e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:57.386796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad833d2-b55c-4ba7-a372-90d2e22f0552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:57.387433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad833d2-b55c-4ba7-a372-90d2e22f0552 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:57.387712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad833d2-b55c-4ba7-a372-90d2e22f0552 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:57.387990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad833d2-b55c-4ba7-a372-90d2e22f0552 None admin] PATH_INFO: `/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:57.389173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad833d2-b55c-4ba7-a372-90d2e22f0552 None admin] RBAC: Authorizing `identity:get_user(user_id=ce768da38f0f4bf58da2a78ad81ab2a4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:57.390703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad833d2-b55c-4ba7-a372-90d2e22f0552 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:57.392707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 132/263] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:57 2022] GET /identity/v3/users/ce768da38f0f4bf58da2a78ad81ab2a4 => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:57.397984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad833d2-b55c-4ba7-a372-90d2e22f0552 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:57.420732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451058dc-8cb8-4498-bf43-a579dc3a85db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:57.422222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451058dc-8cb8-4498-bf43-a579dc3a85db None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:57.536418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:57.549062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad833d2-b55c-4ba7-a372-90d2e22f0552 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:57.604387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413e748-bdf8-4780-bb5d-c5dc081aaa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:57.604984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413e748-bdf8-4780-bb5d-c5dc081aaa72 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:57.605264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413e748-bdf8-4780-bb5d-c5dc081aaa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:57.605541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413e748-bdf8-4780-bb5d-c5dc081aaa72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:57.618947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 132/264] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:49:57.625645 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451058dc-8cb8-4498-bf43-a579dc3a85db None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:57.630487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:57.696782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413e748-bdf8-4780-bb5d-c5dc081aaa72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:57.698286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413e748-bdf8-4780-bb5d-c5dc081aaa72 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:57.775856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451058dc-8cb8-4498-bf43-a579dc3a85db None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:57.858677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e72338-ef3d-487c-81b3-0e2a59d800c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:57.860031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e72338-ef3d-487c-81b3-0e2a59d800c1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:57.860553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e72338-ef3d-487c-81b3-0e2a59d800c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:57.861014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e72338-ef3d-487c-81b3-0e2a59d800c1 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:57.862386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e72338-ef3d-487c-81b3-0e2a59d800c1 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:57.864023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e72338-ef3d-487c-81b3-0e2a59d800c1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:57.866205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 133/265] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:57 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:57.872750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e72338-ef3d-487c-81b3-0e2a59d800c1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:57.897501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:57.911151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 133/266] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:49:57.921732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413e748-bdf8-4780-bb5d-c5dc081aaa72 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:57.992420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:58.044712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e72338-ef3d-487c-81b3-0e2a59d800c1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:58.114051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc9f6a-393c-47cd-8a0e-cb9f15fc39a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:58.115111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc9f6a-393c-47cd-8a0e-cb9f15fc39a4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:58.115582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc9f6a-393c-47cd-8a0e-cb9f15fc39a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:58.116079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc9f6a-393c-47cd-8a0e-cb9f15fc39a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:58.116996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc9f6a-393c-47cd-8a0e-cb9f15fc39a4 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=10e7858e00034a8284453b08e8470748, user.id=ce768da38f0f4bf58da2a78ad81ab2a4) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:58.117514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc9f6a-393c-47cd-8a0e-cb9f15fc39a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:58.119133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc9f6a-393c-47cd-8a0e-cb9f15fc39a4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:58.128930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413e748-bdf8-4780-bb5d-c5dc081aaa72 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:58.134339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 134/267] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:57 2022] GET /identity/v3/role_assignments?user.id=ce768da38f0f4bf58da2a78ad81ab2a4&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:58.187430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5c1b4-02ff-4915-903a-36d496efe5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:58.188106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5c1b4-02ff-4915-903a-36d496efe5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:58.188393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5c1b4-02ff-4915-903a-36d496efe5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:58.188666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5c1b4-02ff-4915-903a-36d496efe5f4 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:58.192722 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5c1b4-02ff-4915-903a-36d496efe5f4 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:58.194208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5c1b4-02ff-4915-903a-36d496efe5f4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:58.196319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 134/268] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:57 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:58.202196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc9f6a-393c-47cd-8a0e-cb9f15fc39a4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:58.348582 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259c684-b53f-43f7-a8ac-4b91c771c2a6 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:58.348925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259c684-b53f-43f7-a8ac-4b91c771c2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:58.349205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259c684-b53f-43f7-a8ac-4b91c771c2a6 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:58.352575 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 135/269] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:58.361483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c2a21-9f22-48df-8f71-163a66230220 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:58.361778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c2a21-9f22-48df-8f71-163a66230220 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:58.362093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c2a21-9f22-48df-8f71-163a66230220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:58.389749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc9f6a-393c-47cd-8a0e-cb9f15fc39a4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:58.397062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-324c2a21-9f22-48df-8f71-163a66230220 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:58.398275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-324c2a21-9f22-48df-8f71-163a66230220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:58.437940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94611f60-e5f0-4721-b7e0-75a54f76df26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:58.440590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94611f60-e5f0-4721-b7e0-75a54f76df26 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:58.441300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94611f60-e5f0-4721-b7e0-75a54f76df26 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:58.442554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94611f60-e5f0-4721-b7e0-75a54f76df26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:58.443996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94611f60-e5f0-4721-b7e0-75a54f76df26 None admin] RBAC: Adding query filter params (role.id=fc1186ecd0f74b2fad8155d528f268f9, scope.project.id=10e7858e00034a8284453b08e8470748, user.id=ce768da38f0f4bf58da2a78ad81ab2a4) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:58.444727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94611f60-e5f0-4721-b7e0-75a54f76df26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:58.447275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94611f60-e5f0-4721-b7e0-75a54f76df26 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:58.460423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:58.464993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 135/270] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:58 2022] GET /identity/v3/role_assignments?user.id=ce768da38f0f4bf58da2a78ad81ab2a4&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=fc1186ecd0f74b2fad8155d528f268f9 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:58.507720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:58.519732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 136/271] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:58.541733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd55a89-c1a7-4ac7-8aeb-ab8cd9410e32 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:58.542090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd55a89-c1a7-4ac7-8aeb-ab8cd9410e32 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:58.542332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd55a89-c1a7-4ac7-8aeb-ab8cd9410e32 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:58.543923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 136/272] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:58.551218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832628e-8cd1-4ad5-af2f-71e6333f3980 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:58.551697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832628e-8cd1-4ad5-af2f-71e6333f3980 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:58.551769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832628e-8cd1-4ad5-af2f-71e6333f3980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:58.577110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b832628e-8cd1-4ad5-af2f-71e6333f3980 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:58.578291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b832628e-8cd1-4ad5-af2f-71e6333f3980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:58.663188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 137/273] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:58.837310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd55a89-c1a7-4ac7-8aeb-ab8cd9410e32 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:58.887528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dcc14-4206-4dbb-9ad4-b09f2ee7bc44 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:58.887888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dcc14-4206-4dbb-9ad4-b09f2ee7bc44 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:58.888387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dcc14-4206-4dbb-9ad4-b09f2ee7bc44 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:58.889887 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 138/274] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:49:58.896253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44729ac7-5f23-493e-b2a7-b50116534030 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:58.896767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44729ac7-5f23-493e-b2a7-b50116534030 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:58.897269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44729ac7-5f23-493e-b2a7-b50116534030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:58.920982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-44729ac7-5f23-493e-b2a7-b50116534030 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:58.922552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-44729ac7-5f23-493e-b2a7-b50116534030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:58.963719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd55a89-c1a7-4ac7-8aeb-ab8cd9410e32 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:59.011102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dab09-c8b5-49ba-b335-4f39c39a7720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:59.011787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dab09-c8b5-49ba-b335-4f39c39a7720 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.011877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dab09-c8b5-49ba-b335-4f39c39a7720 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.012120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dab09-c8b5-49ba-b335-4f39c39a7720 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.026678 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 139/275] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:49:59.027553 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44729ac7-5f23-493e-b2a7-b50116534030 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:59.027916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0dab09-c8b5-49ba-b335-4f39c39a7720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-dc0dab09-c8b5-49ba-b335-4f39c39a7720 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:59.033203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:59.034609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:49:59.035403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 137/276] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:59.046310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dab09-c8b5-49ba-b335-4f39c39a7720 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:59.112710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:59.124646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:59.161082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44729ac7-5f23-493e-b2a7-b50116534030 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:59.171622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dab09-c8b5-49ba-b335-4f39c39a7720 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:59.197993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e4251-90c1-4aa1-a10f-f20ed0e48d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:59.198763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e4251-90c1-4aa1-a10f-f20ed0e48d46 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.199044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e4251-90c1-4aa1-a10f-f20ed0e48d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.199320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e4251-90c1-4aa1-a10f-f20ed0e48d46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.200457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e4251-90c1-4aa1-a10f-f20ed0e48d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:59.202144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e4251-90c1-4aa1-a10f-f20ed0e48d46 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:59.203732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 140/277] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:49:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:59.205020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c3336-7431-4534-92a2-8f9b11aa588e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.205308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c3336-7431-4534-92a2-8f9b11aa588e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.205586 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c3336-7431-4534-92a2-8f9b11aa588e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.207206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10257dd4-48f6-47d7-9385-6f7ea819f10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:59.207805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10257dd4-48f6-47d7-9385-6f7ea819f10d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.208827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10257dd4-48f6-47d7-9385-6f7ea819f10d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.208827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10257dd4-48f6-47d7-9385-6f7ea819f10d None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.209106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10257dd4-48f6-47d7-9385-6f7ea819f10d None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:59.209455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10257dd4-48f6-47d7-9385-6f7ea819f10d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:59.210837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10257dd4-48f6-47d7-9385-6f7ea819f10d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:59.211715 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 141/278] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:49:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:49:59.212447 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c3336-7431-4534-92a2-8f9b11aa588e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:59.225625 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 138/279] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:59.231347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce20b-9e7e-458d-94e6-e1c71637f574 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.231637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce20b-9e7e-458d-94e6-e1c71637f574 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.231909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce20b-9e7e-458d-94e6-e1c71637f574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.249433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fddce20b-9e7e-458d-94e6-e1c71637f574 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:59.250598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fddce20b-9e7e-458d-94e6-e1c71637f574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:59.318215 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 139/280] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:49:59.319072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddce20b-9e7e-458d-94e6-e1c71637f574 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:59.324837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c3336-7431-4534-92a2-8f9b11aa588e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:59.349797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:59.363783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502facc8-cf8f-43d9-a6d7-ff12d7c85f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:59.364414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502facc8-cf8f-43d9-a6d7-ff12d7c85f3d None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.364815 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502facc8-cf8f-43d9-a6d7-ff12d7c85f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.365093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502facc8-cf8f-43d9-a6d7-ff12d7c85f3d None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.365739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502facc8-cf8f-43d9-a6d7-ff12d7c85f3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:59.367350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502facc8-cf8f-43d9-a6d7-ff12d7c85f3d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:59.374567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:59.400078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 142/281] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:49:59 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:49:59.400078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502facc8-cf8f-43d9-a6d7-ff12d7c85f3d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:59.445423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddce20b-9e7e-458d-94e6-e1c71637f574 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:59.495151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1be0c5-56e9-4fbb-b724-acdc49e307e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:59.495933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1be0c5-56e9-4fbb-b724-acdc49e307e0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.496324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1be0c5-56e9-4fbb-b724-acdc49e307e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.497018 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1be0c5-56e9-4fbb-b724-acdc49e307e0 None admin] PATH_INFO: `/v3/users/75c1fdec00b64c569072ce5d258cc6e6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.516949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1be0c5-56e9-4fbb-b724-acdc49e307e0 None admin] RBAC: Authorizing `identity:get_user(user_id=75c1fdec00b64c569072ce5d258cc6e6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:59.519959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1be0c5-56e9-4fbb-b724-acdc49e307e0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:59.522054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 140/282] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:59 2022] GET /identity/v3/users/75c1fdec00b64c569072ce5d258cc6e6 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:59.522185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502facc8-cf8f-43d9-a6d7-ff12d7c85f3d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:59.533121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1be0c5-56e9-4fbb-b724-acdc49e307e0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:59.575029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e75d48-4457-4c91-afa5-8838c9e1c1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:59.575677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e75d48-4457-4c91-afa5-8838c9e1c1ae None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.575959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e75d48-4457-4c91-afa5-8838c9e1c1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.577117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e75d48-4457-4c91-afa5-8838c9e1c1ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.598248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e75d48-4457-4c91-afa5-8838c9e1c1ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-20e75d48-4457-4c91-afa5-8838c9e1c1ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:59.603488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:59.604438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:59.605453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 143/283] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:59.606360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e75d48-4457-4c91-afa5-8838c9e1c1ae None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:59.653763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:59.679981 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1be0c5-56e9-4fbb-b724-acdc49e307e0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:59.697400 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:59.733310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217ba61-876a-43e8-9a72-b4db4b1acccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:59.733928 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217ba61-876a-43e8-9a72-b4db4b1acccb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.734211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217ba61-876a-43e8-9a72-b4db4b1acccb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.734487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217ba61-876a-43e8-9a72-b4db4b1acccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.736077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217ba61-876a-43e8-9a72-b4db4b1acccb None admin] RBAC: Adding query filter params (role.id=7ab2687826544b8e80c709d6061162d4, user.id=75c1fdec00b64c569072ce5d258cc6e6, scope.system=all) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:59.736077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217ba61-876a-43e8-9a72-b4db4b1acccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:59.737011 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217ba61-876a-43e8-9a72-b4db4b1acccb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:59.742219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e75d48-4457-4c91-afa5-8838c9e1c1ae None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:59.743214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 141/284] 10.222.0.86 () {56 vars in 1308 bytes} [Wed Aug 24 09:49:59 2022] GET /identity/v3/role_assignments?user.id=75c1fdec00b64c569072ce5d258cc6e6&scope.system=all&role.id=7ab2687826544b8e80c709d6061162d4 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:59.744224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217ba61-876a-43e8-9a72-b4db4b1acccb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:59.781221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ea7b3-4912-4bd8-9a8f-f9f12fb09b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:59.781874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ea7b3-4912-4bd8-9a8f-f9f12fb09b79 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.782157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ea7b3-4912-4bd8-9a8f-f9f12fb09b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.782432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ea7b3-4912-4bd8-9a8f-f9f12fb09b79 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.795092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ea7b3-4912-4bd8-9a8f-f9f12fb09b79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-252ea7b3-4912-4bd8-9a8f-f9f12fb09b79 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:59.801008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:59.802164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:49:59.807985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 144/285] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:59.812644 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ea7b3-4912-4bd8-9a8f-f9f12fb09b79 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:59.879899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217ba61-876a-43e8-9a72-b4db4b1acccb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:59.918902 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bb71d-a527-4c70-85ae-9db4c92ed1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:59.919697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bb71d-a527-4c70-85ae-9db4c92ed1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.920100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bb71d-a527-4c70-85ae-9db4c92ed1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.920482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bb71d-a527-4c70-85ae-9db4c92ed1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.921271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bb71d-a527-4c70-85ae-9db4c92ed1d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:59.921685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bb71d-a527-4c70-85ae-9db4c92ed1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:59.923239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bb71d-a527-4c70-85ae-9db4c92ed1d8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:59.929291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:59.934363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 142/286] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:59.934823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ea7b3-4912-4bd8-9a8f-f9f12fb09b79 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:59.939785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bb71d-a527-4c70-85ae-9db4c92ed1d8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:59.974470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b9abe-7b1a-4c5e-a6c3-d6d3f09cffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:59.975353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b9abe-7b1a-4c5e-a6c3-d6d3f09cffc8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.975794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b9abe-7b1a-4c5e-a6c3-d6d3f09cffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.976269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b9abe-7b1a-4c5e-a6c3-d6d3f09cffc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.977128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b9abe-7b1a-4c5e-a6c3-d6d3f09cffc8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:59.977624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b9abe-7b1a-4c5e-a6c3-d6d3f09cffc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:59.979287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b9abe-7b1a-4c5e-a6c3-d6d3f09cffc8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:59.993617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 145/287] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:49:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:49:59.995116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:59.996910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b9abe-7b1a-4c5e-a6c3-d6d3f09cffc8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:00.087472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bb71d-a527-4c70-85ae-9db4c92ed1d8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:00.133477 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b64c7-21bc-4f3f-84ea-6bb3c8928236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:00.134120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b64c7-21bc-4f3f-84ea-6bb3c8928236 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:00.134401 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b64c7-21bc-4f3f-84ea-6bb3c8928236 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:00.134676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b64c7-21bc-4f3f-84ea-6bb3c8928236 None admin] PATH_INFO: `/v3/users/ac1b33d6e6434973bdb8f3883c988035` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:00.135930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b64c7-21bc-4f3f-84ea-6bb3c8928236 None admin] RBAC: Authorizing `identity:get_user(user_id=ac1b33d6e6434973bdb8f3883c988035)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:00.137434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b64c7-21bc-4f3f-84ea-6bb3c8928236 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:00.139062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 143/288] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:59 2022] GET /identity/v3/users/ac1b33d6e6434973bdb8f3883c988035 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:00.147202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b64c7-21bc-4f3f-84ea-6bb3c8928236 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:00.173632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b9abe-7b1a-4c5e-a6c3-d6d3f09cffc8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:00.246387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148450e-6a33-49ca-a23c-561b1b4fa63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:00.246948 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148450e-6a33-49ca-a23c-561b1b4fa63f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:00.247232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148450e-6a33-49ca-a23c-561b1b4fa63f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:00.247504 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148450e-6a33-49ca-a23c-561b1b4fa63f None admin] PATH_INFO: `/v3/users/c1f0fd904ad74d8ba3c262518f23a93c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:00.249643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148450e-6a33-49ca-a23c-561b1b4fa63f None admin] RBAC: Authorizing `identity:get_user(user_id=c1f0fd904ad74d8ba3c262518f23a93c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:00.251152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148450e-6a33-49ca-a23c-561b1b4fa63f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:00.254478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 146/289] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:59 2022] GET /identity/v3/users/c1f0fd904ad74d8ba3c262518f23a93c => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:00.262259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148450e-6a33-49ca-a23c-561b1b4fa63f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:00.295214 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:00.342554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b64c7-21bc-4f3f-84ea-6bb3c8928236 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:00.350574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:00.380472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769372a-24cb-4523-9308-dd5a4cd2790c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:00.381210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769372a-24cb-4523-9308-dd5a4cd2790c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:00.381491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769372a-24cb-4523-9308-dd5a4cd2790c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:00.381766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769372a-24cb-4523-9308-dd5a4cd2790c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:00.422905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148450e-6a33-49ca-a23c-561b1b4fa63f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:00.460999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769372a-24cb-4523-9308-dd5a4cd2790c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:00.467920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769372a-24cb-4523-9308-dd5a4cd2790c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:00.485614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b198fce-ba40-481a-8cbd-e23366d3c723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:00.486246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b198fce-ba40-481a-8cbd-e23366d3c723 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:00.486522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b198fce-ba40-481a-8cbd-e23366d3c723 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:00.486794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b198fce-ba40-481a-8cbd-e23366d3c723 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:00.492713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b198fce-ba40-481a-8cbd-e23366d3c723 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:00.494154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b198fce-ba40-481a-8cbd-e23366d3c723 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:00.496255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 147/290] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:00 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:00.501903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b198fce-ba40-481a-8cbd-e23366d3c723 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:00.570943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 144/291] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:00.585088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769372a-24cb-4523-9308-dd5a4cd2790c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:00.598728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:00.664405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:00.671707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b198fce-ba40-481a-8cbd-e23366d3c723 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:00.739765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3d520-e270-46b3-9ee4-508a7fdc3e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:00.740446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3d520-e270-46b3-9ee4-508a7fdc3e3f None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:00.740722 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3d520-e270-46b3-9ee4-508a7fdc3e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:00.740992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3d520-e270-46b3-9ee4-508a7fdc3e3f None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748/users/c1f0fd904ad74d8ba3c262518f23a93c/roles/7fcd054cc7c549b18e4ac97d4a405e8c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:00.743746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3d520-e270-46b3-9ee4-508a7fdc3e3f None admin] RBAC: Authorizing `identity:create_grant(project_id=10e7858e00034a8284453b08e8470748, user_id=c1f0fd904ad74d8ba3c262518f23a93c, role_id=7fcd054cc7c549b18e4ac97d4a405e8c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:00.745452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3d520-e270-46b3-9ee4-508a7fdc3e3f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:00.770350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769372a-24cb-4523-9308-dd5a4cd2790c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:00.775171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 148/292] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:50:00 2022] PUT /identity/v3/projects/10e7858e00034a8284453b08e8470748/users/c1f0fd904ad74d8ba3c262518f23a93c/roles/7fcd054cc7c549b18e4ac97d4a405e8c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:50:00.823747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c97169-d84b-4de4-8a91-14f78cfca5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:00.824481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c97169-d84b-4de4-8a91-14f78cfca5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:00.824762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c97169-d84b-4de4-8a91-14f78cfca5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:00.825038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c97169-d84b-4de4-8a91-14f78cfca5f8 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:00.828874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c97169-d84b-4de4-8a91-14f78cfca5f8 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:00.830342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c97169-d84b-4de4-8a91-14f78cfca5f8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:00.832463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 145/293] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:00 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:00.838336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3d520-e270-46b3-9ee4-508a7fdc3e3f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:01.018735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3d520-e270-46b3-9ee4-508a7fdc3e3f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:01.081719 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfd540-3b8d-48f5-a1d4-cba0f375a2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:01.082641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfd540-3b8d-48f5-a1d4-cba0f375a2cf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:01.083118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfd540-3b8d-48f5-a1d4-cba0f375a2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:01.083591 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfd540-3b8d-48f5-a1d4-cba0f375a2cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:01.085024 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfd540-3b8d-48f5-a1d4-cba0f375a2cf None admin] RBAC: Adding query filter params (role.id=7ab2687826544b8e80c709d6061162d4, scope.project.id=10e7858e00034a8284453b08e8470748, user.id=ac1b33d6e6434973bdb8f3883c988035) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:01.085563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfd540-3b8d-48f5-a1d4-cba0f375a2cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:01.088208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfd540-3b8d-48f5-a1d4-cba0f375a2cf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:01.103884 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 149/294] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:00 2022] GET /identity/v3/role_assignments?user.id=ac1b33d6e6434973bdb8f3883c988035&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=7ab2687826544b8e80c709d6061162d4 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:01.768094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb10fa-7c06-4614-8c84-28883b96fe3b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:01.769219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb10fa-7c06-4614-8c84-28883b96fe3b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:01.769870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb10fa-7c06-4614-8c84-28883b96fe3b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:01.771708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 146/295] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:01.777668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4fa62-455f-43c9-bf58-26ae2744fffa None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:01.778113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4fa62-455f-43c9-bf58-26ae2744fffa None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:01.778210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4fa62-455f-43c9-bf58-26ae2744fffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:01.805080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e3d4fa62-455f-43c9-bf58-26ae2744fffa None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:01.806586 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e3d4fa62-455f-43c9-bf58-26ae2744fffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:01.822346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:01.884167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 150/296] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:02.096749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0673037-32cd-495d-89be-e73b153c769d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:02.097474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0673037-32cd-495d-89be-e73b153c769d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:02.097992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0673037-32cd-495d-89be-e73b153c769d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:02.099787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 147/297] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:02.105333 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2bd10-3da3-4902-89f0-d51572210406 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:02.105851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2bd10-3da3-4902-89f0-d51572210406 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:02.106345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2bd10-3da3-4902-89f0-d51572210406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:02.111450 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e45b4-f20e-4394-8489-f35bfd3d98d7 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:02.111977 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e45b4-f20e-4394-8489-f35bfd3d98d7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:02.112566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e45b4-f20e-4394-8489-f35bfd3d98d7 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:02.114576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 148/298] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:50:02.119793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649ea6b-a405-4635-bacf-2f8cfacbbad6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:02.120106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649ea6b-a405-4635-bacf-2f8cfacbbad6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:02.120395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649ea6b-a405-4635-bacf-2f8cfacbbad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:02.126338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e5e2bd10-3da3-4902-89f0-d51572210406 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:02.127767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e5e2bd10-3da3-4902-89f0-d51572210406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:02.138565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8649ea6b-a405-4635-bacf-2f8cfacbbad6 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:02.139961 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8649ea6b-a405-4635-bacf-2f8cfacbbad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:02.204358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 151/299] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:02.205110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2bd10-3da3-4902-89f0-d51572210406 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:02.219364 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 149/300] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:02.304469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:02.324717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2bd10-3da3-4902-89f0-d51572210406 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:02.360973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbbe70-9238-484e-950c-438afc79ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:02.361522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbbe70-9238-484e-950c-438afc79ed24 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:02.361802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbbe70-9238-484e-950c-438afc79ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:02.362074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbbe70-9238-484e-950c-438afc79ed24 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:02.372808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbbe70-9238-484e-950c-438afc79ed24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-3cbbbe70-9238-484e-950c-438afc79ed24 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:02.377609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:02.378526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:02.379332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 152/301] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:02.384530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649ea6b-a405-4635-bacf-2f8cfacbbad6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:02.388912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:02.480823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbbe70-9238-484e-950c-438afc79ed24 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:02.494476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649ea6b-a405-4635-bacf-2f8cfacbbad6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:02.531656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e57f9-5ad1-40fc-817c-ed9d0dc64168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:02.532630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e57f9-5ad1-40fc-817c-ed9d0dc64168 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:02.533134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e57f9-5ad1-40fc-817c-ed9d0dc64168 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:02.533615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e57f9-5ad1-40fc-817c-ed9d0dc64168 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:02.535095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e57f9-5ad1-40fc-817c-ed9d0dc64168 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:02.535642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e57f9-5ad1-40fc-817c-ed9d0dc64168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:02.537233 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e57f9-5ad1-40fc-817c-ed9d0dc64168 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:02.546416 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 150/302] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:02.546416 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e57f9-5ad1-40fc-817c-ed9d0dc64168 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:02.592663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbbe70-9238-484e-950c-438afc79ed24 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:02.615962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:02.629935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c274-12da-4336-b8f6-f6de8488e22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:02.630571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c274-12da-4336-b8f6-f6de8488e22d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:02.630854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c274-12da-4336-b8f6-f6de8488e22d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:02.631124 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c274-12da-4336-b8f6-f6de8488e22d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:02.632273 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3c274-12da-4336-b8f6-f6de8488e22d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:02.633733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3c274-12da-4336-b8f6-f6de8488e22d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:02.635489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 153/303] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:02.636079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c274-12da-4336-b8f6-f6de8488e22d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:02.649006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:02.663408 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e57f9-5ad1-40fc-817c-ed9d0dc64168 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:02.699098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e81591-ddd9-407a-8022-e5bf626cf735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:02.699735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e81591-ddd9-407a-8022-e5bf626cf735 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:02.700014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e81591-ddd9-407a-8022-e5bf626cf735 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:02.700312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e81591-ddd9-407a-8022-e5bf626cf735 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:02.711231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e81591-ddd9-407a-8022-e5bf626cf735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b5e81591-ddd9-407a-8022-e5bf626cf735 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:02.717061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:02.718000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:02.719120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 151/304] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:02.720001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e81591-ddd9-407a-8022-e5bf626cf735 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:02.751943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c274-12da-4336-b8f6-f6de8488e22d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:02.789082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fe5af-bb93-4e06-a419-a07175a677cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:02.789879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fe5af-bb93-4e06-a419-a07175a677cb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:02.790102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fe5af-bb93-4e06-a419-a07175a677cb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:02.790369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fe5af-bb93-4e06-a419-a07175a677cb None admin] PATH_INFO: `/v3/users/3431213ea7d74427b80e1ddd6fdf1398` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:02.805190 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fe5af-bb93-4e06-a419-a07175a677cb None admin] RBAC: Authorizing `identity:get_user(user_id=3431213ea7d74427b80e1ddd6fdf1398)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:02.806647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fe5af-bb93-4e06-a419-a07175a677cb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:02.813271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 154/305] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:50:02 2022] GET /identity/v3/users/3431213ea7d74427b80e1ddd6fdf1398 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:02.814072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fe5af-bb93-4e06-a419-a07175a677cb None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:02.832259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e81591-ddd9-407a-8022-e5bf626cf735 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:02.857178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:02.868949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e1219-441a-45b2-88db-50211e442f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:02.869498 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e1219-441a-45b2-88db-50211e442f6a None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:02.869777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e1219-441a-45b2-88db-50211e442f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:02.870053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e1219-441a-45b2-88db-50211e442f6a None admin] PATH_INFO: `/v3/groups` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:02.870685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e1219-441a-45b2-88db-50211e442f6a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:02.872142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e1219-441a-45b2-88db-50211e442f6a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:02.889767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 152/306] 10.222.0.86 () {60 vars in 1134 bytes} [Wed Aug 24 09:50:02 2022] POST /identity/v3/groups => generated 229 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:02.890849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:02.901095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e1219-441a-45b2-88db-50211e442f6a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:02.933456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fe5af-bb93-4e06-a419-a07175a677cb None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:02.983245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecad4e0-2a3c-481b-822a-8ad815c0ace4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:02.983946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecad4e0-2a3c-481b-822a-8ad815c0ace4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:02.984367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecad4e0-2a3c-481b-822a-8ad815c0ace4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:02.984680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecad4e0-2a3c-481b-822a-8ad815c0ace4 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:02.985456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecad4e0-2a3c-481b-822a-8ad815c0ace4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:02.985845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecad4e0-2a3c-481b-822a-8ad815c0ace4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:02.987490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecad4e0-2a3c-481b-822a-8ad815c0ace4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:03.004171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 155/307] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:03.005204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecad4e0-2a3c-481b-822a-8ad815c0ace4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:03.054206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e1219-441a-45b2-88db-50211e442f6a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:03.103180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194ba21-8c2b-42d0-ab7c-755af4fb798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:03.103799 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194ba21-8c2b-42d0-ab7c-755af4fb798f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:03.104153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194ba21-8c2b-42d0-ab7c-755af4fb798f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:03.104431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194ba21-8c2b-42d0-ab7c-755af4fb798f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:03.105171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194ba21-8c2b-42d0-ab7c-755af4fb798f None admin] RBAC: Adding query filter params (role.id=7fcd054cc7c549b18e4ac97d4a405e8c, user.id=3431213ea7d74427b80e1ddd6fdf1398, scope.system=all) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:03.105485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194ba21-8c2b-42d0-ab7c-755af4fb798f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:03.106951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194ba21-8c2b-42d0-ab7c-755af4fb798f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:03.113535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 153/308] 10.222.0.86 () {56 vars in 1308 bytes} [Wed Aug 24 09:50:02 2022] GET /identity/v3/role_assignments?user.id=3431213ea7d74427b80e1ddd6fdf1398&scope.system=all&role.id=7fcd054cc7c549b18e4ac97d4a405e8c => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:03.143784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecad4e0-2a3c-481b-822a-8ad815c0ace4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:03.156453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371095c-7be7-4a36-9f3e-c81603b85b72 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:03.156898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371095c-7be7-4a36-9f3e-c81603b85b72 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:03.157280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371095c-7be7-4a36-9f3e-c81603b85b72 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:03.159148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 154/309] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:50:03.164939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611e09b-913c-4d57-acac-a07d57d76b33 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:03.165338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611e09b-913c-4d57-acac-a07d57d76b33 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:03.165711 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611e09b-913c-4d57-acac-a07d57d76b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:03.178921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:03.186231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9611e09b-913c-4d57-acac-a07d57d76b33 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:03.187609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9611e09b-913c-4d57-acac-a07d57d76b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:03.189921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829b48f-c35f-44ae-b490-e159000bb009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:03.190823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829b48f-c35f-44ae-b490-e159000bb009 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:03.191894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829b48f-c35f-44ae-b490-e159000bb009 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:03.192463 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829b48f-c35f-44ae-b490-e159000bb009 None admin] PATH_INFO: `/v3/users/75c1fdec00b64c569072ce5d258cc6e6` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:03.194005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829b48f-c35f-44ae-b490-e159000bb009 None admin] RBAC: Authorizing `identity:get_user(user_id=75c1fdec00b64c569072ce5d258cc6e6)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:03.195864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829b48f-c35f-44ae-b490-e159000bb009 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:03.198182 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 156/310] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:50:03 2022] GET /identity/v3/users/75c1fdec00b64c569072ce5d258cc6e6 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:03.204659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829b48f-c35f-44ae-b490-e159000bb009 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:03.247165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:03.281595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 155/311] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:03.325437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829b48f-c35f-44ae-b490-e159000bb009 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:03.360280 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e7c68-3a9c-464c-b149-9e6e2be9aecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:03.360931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e7c68-3a9c-464c-b149-9e6e2be9aecc None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:03.361215 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e7c68-3a9c-464c-b149-9e6e2be9aecc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:03.361492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e7c68-3a9c-464c-b149-9e6e2be9aecc None admin] PATH_INFO: `/v3/system/users/75c1fdec00b64c569072ce5d258cc6e6/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:03.363444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e7c68-3a9c-464c-b149-9e6e2be9aecc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=75c1fdec00b64c569072ce5d258cc6e6, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:03.364872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e7c68-3a9c-464c-b149-9e6e2be9aecc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:03.374808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 157/312] 10.222.0.86 () {58 vars in 1334 bytes} [Wed Aug 24 09:50:03 2022] PUT /identity/v3/system/users/75c1fdec00b64c569072ce5d258cc6e6/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:50:03.558526 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611e09b-913c-4d57-acac-a07d57d76b33 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:03.661796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611e09b-913c-4d57-acac-a07d57d76b33 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:03.696828 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d959cc-875b-44eb-aca0-f9c1259f3a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:03.697456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d959cc-875b-44eb-aca0-f9c1259f3a80 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:03.697736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d959cc-875b-44eb-aca0-f9c1259f3a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:03.698005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d959cc-875b-44eb-aca0-f9c1259f3a80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:03.708369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d959cc-875b-44eb-aca0-f9c1259f3a80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c2d959cc-875b-44eb-aca0-f9c1259f3a80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:03.713084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:03.713966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:03.714756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 156/313] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:03.720307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e7c68-3a9c-464c-b149-9e6e2be9aecc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:03.785812 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:03.832090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e7c68-3a9c-464c-b149-9e6e2be9aecc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:03.867172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd91073-e98e-4953-be58-37857d3e8fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:03.867776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd91073-e98e-4953-be58-37857d3e8fba None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:03.868074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd91073-e98e-4953-be58-37857d3e8fba None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:03.868348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd91073-e98e-4953-be58-37857d3e8fba None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:03.869017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd91073-e98e-4953-be58-37857d3e8fba None admin] RBAC: Adding query filter params (name=reader) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:03.869321 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd91073-e98e-4953-be58-37857d3e8fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:03.870706 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd91073-e98e-4953-be58-37857d3e8fba None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:03.879649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 158/314] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:03.885183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d959cc-875b-44eb-aca0-f9c1259f3a80 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:03.927571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:03.990247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d959cc-875b-44eb-aca0-f9c1259f3a80 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:04.025816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcaba5c-5765-4a06-9a5d-9ec929bdce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:04.026445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaba5c-5765-4a06-9a5d-9ec929bdce99 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:04.026721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaba5c-5765-4a06-9a5d-9ec929bdce99 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:04.026993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcaba5c-5765-4a06-9a5d-9ec929bdce99 None admin] PATH_INFO: `/v3/users/c1f0fd904ad74d8ba3c262518f23a93c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:04.028163 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcaba5c-5765-4a06-9a5d-9ec929bdce99 None admin] RBAC: Authorizing `identity:get_user(user_id=c1f0fd904ad74d8ba3c262518f23a93c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:04.029619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcaba5c-5765-4a06-9a5d-9ec929bdce99 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:04.031529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 157/315] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:50:03 2022] GET /identity/v3/users/c1f0fd904ad74d8ba3c262518f23a93c => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:04.037784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd91073-e98e-4953-be58-37857d3e8fba None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:04.156235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd91073-e98e-4953-be58-37857d3e8fba None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:04.180426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:04.192732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a326569-80c5-4242-858d-0b94e596f8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:04.193338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a326569-80c5-4242-858d-0b94e596f8ad None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:04.193613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a326569-80c5-4242-858d-0b94e596f8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:04.193882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a326569-80c5-4242-858d-0b94e596f8ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:04.267077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a326569-80c5-4242-858d-0b94e596f8ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:04.268588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a326569-80c5-4242-858d-0b94e596f8ad None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:04.369966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 159/316] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:04.375529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcaba5c-5765-4a06-9a5d-9ec929bdce99 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:04.476816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcaba5c-5765-4a06-9a5d-9ec929bdce99 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:04.484513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:04.513728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055d9f5-41c6-43cd-a11b-ef278112c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:04.514360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055d9f5-41c6-43cd-a11b-ef278112c500 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:04.514655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055d9f5-41c6-43cd-a11b-ef278112c500 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:04.514905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055d9f5-41c6-43cd-a11b-ef278112c500 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:04.516022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055d9f5-41c6-43cd-a11b-ef278112c500 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:04.517500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055d9f5-41c6-43cd-a11b-ef278112c500 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:04.519532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 158/317] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:04 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:04.525762 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a326569-80c5-4242-858d-0b94e596f8ad None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:04.552614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:04.633187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a326569-80c5-4242-858d-0b94e596f8ad None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:04.669053 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfd3a1-71fc-47ce-8766-de27c8d671b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:04.669649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfd3a1-71fc-47ce-8766-de27c8d671b7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:04.669925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfd3a1-71fc-47ce-8766-de27c8d671b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:04.670250 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfd3a1-71fc-47ce-8766-de27c8d671b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:04.670974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfd3a1-71fc-47ce-8766-de27c8d671b7 None admin] RBAC: Adding query filter params (role.id=7fcd054cc7c549b18e4ac97d4a405e8c, scope.project.id=10e7858e00034a8284453b08e8470748, user.id=c1f0fd904ad74d8ba3c262518f23a93c) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:04.671325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfd3a1-71fc-47ce-8766-de27c8d671b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:04.672798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfd3a1-71fc-47ce-8766-de27c8d671b7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:04.678614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 160/318] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:04 2022] GET /identity/v3/role_assignments?user.id=c1f0fd904ad74d8ba3c262518f23a93c&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=7fcd054cc7c549b18e4ac97d4a405e8c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:04.949619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4cf00-7d35-40b1-ab70-d036c41aec65 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:04.950132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4cf00-7d35-40b1-ab70-d036c41aec65 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:04.950206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4cf00-7d35-40b1-ab70-d036c41aec65 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:04.951659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 159/319] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:04.956836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181b626-14f2-4808-a481-93284d5d1315 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:04.957217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181b626-14f2-4808-a481-93284d5d1315 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:04.957384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181b626-14f2-4808-a481-93284d5d1315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:04.976088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6181b626-14f2-4808-a481-93284d5d1315 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:04.977296 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6181b626-14f2-4808-a481-93284d5d1315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:05.048666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 161/320] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:05.079076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac133cff-ca86-47eb-b812-62936391fd24 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.079378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac133cff-ca86-47eb-b812-62936391fd24 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.079627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac133cff-ca86-47eb-b812-62936391fd24 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.081131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 160/321] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:05.086231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9238f0-1ac2-49b2-9a30-8d0fddac6b13 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.086515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9238f0-1ac2-49b2-9a30-8d0fddac6b13 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.086781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9238f0-1ac2-49b2-9a30-8d0fddac6b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.105610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ce9238f0-1ac2-49b2-9a30-8d0fddac6b13 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:05.106978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ce9238f0-1ac2-49b2-9a30-8d0fddac6b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:05.118978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:05.179090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 162/322] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:05.320089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac133cff-ca86-47eb-b812-62936391fd24 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:05.423364 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac133cff-ca86-47eb-b812-62936391fd24 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:05.441741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaca9db-2892-46c4-95ee-a066588f0095 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.442037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaca9db-2892-46c4-95ee-a066588f0095 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.442300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaca9db-2892-46c4-95ee-a066588f0095 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.443663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 163/323] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:50:05.448561 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ed316-8269-49ac-ae29-7b7b894836ef None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.448849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ed316-8269-49ac-ae29-7b7b894836ef None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.449171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ed316-8269-49ac-ae29-7b7b894836ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.458783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83399a2-15a0-475c-9209-74f5c8393f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:05.459411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83399a2-15a0-475c-9209-74f5c8393f4e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.459692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83399a2-15a0-475c-9209-74f5c8393f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.459966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83399a2-15a0-475c-9209-74f5c8393f4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.461103 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83399a2-15a0-475c-9209-74f5c8393f4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:05.462544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83399a2-15a0-475c-9209-74f5c8393f4e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:05.464140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 161/324] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:05.466103 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83399a2-15a0-475c-9209-74f5c8393f4e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:05.469285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2c5ed316-8269-49ac-ae29-7b7b894836ef None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:05.470480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2c5ed316-8269-49ac-ae29-7b7b894836ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:05.508699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:05.537227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 164/325] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:05.538054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ed316-8269-49ac-ae29-7b7b894836ef None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:05.572378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83399a2-15a0-475c-9209-74f5c8393f4e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:05.610677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77fc9d-c6b0-427f-b893-240ccafcf312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:05.611299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77fc9d-c6b0-427f-b893-240ccafcf312 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.611577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77fc9d-c6b0-427f-b893-240ccafcf312 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.611852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77fc9d-c6b0-427f-b893-240ccafcf312 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.622091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77fc9d-c6b0-427f-b893-240ccafcf312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:05.622930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8b77fc9d-c6b0-427f-b893-240ccafcf312 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:05.626822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:05.627712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:05.628562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 162/326] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:05.633642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77fc9d-c6b0-427f-b893-240ccafcf312 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:05.652777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ed316-8269-49ac-ae29-7b7b894836ef None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:05.691031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc946224-a468-4421-b3a0-0500727dc92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:05.692565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc946224-a468-4421-b3a0-0500727dc92f None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.692995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc946224-a468-4421-b3a0-0500727dc92f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.693416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc946224-a468-4421-b3a0-0500727dc92f None admin] PATH_INFO: `/v3/groups` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.694296 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc946224-a468-4421-b3a0-0500727dc92f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:05.696005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc946224-a468-4421-b3a0-0500727dc92f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:05.715228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 165/327] 10.222.0.86 () {60 vars in 1134 bytes} [Wed Aug 24 09:50:05 2022] POST /identity/v3/groups => generated 226 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:05.740675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77fc9d-c6b0-427f-b893-240ccafcf312 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:05.744394 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:05.787205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa2c6b-4244-48ef-a3bb-898f533968bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:05.787870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2c6b-4244-48ef-a3bb-898f533968bc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.788179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2c6b-4244-48ef-a3bb-898f533968bc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.788489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2c6b-4244-48ef-a3bb-898f533968bc None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.789178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa2c6b-4244-48ef-a3bb-898f533968bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:05.789479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa2c6b-4244-48ef-a3bb-898f533968bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:05.790886 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa2c6b-4244-48ef-a3bb-898f533968bc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:05.805648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 163/328] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:05.806926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc946224-a468-4421-b3a0-0500727dc92f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:05.817797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa2c6b-4244-48ef-a3bb-898f533968bc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:05.919401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc946224-a468-4421-b3a0-0500727dc92f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:05.942676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa2c6b-4244-48ef-a3bb-898f533968bc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:05.967767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffd67b-ceb0-4d6f-84d3-c03d94238935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:05.969320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffd67b-ceb0-4d6f-84d3-c03d94238935 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.969751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffd67b-ceb0-4d6f-84d3-c03d94238935 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.970171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffd67b-ceb0-4d6f-84d3-c03d94238935 None admin] PATH_INFO: `/v3/users/3431213ea7d74427b80e1ddd6fdf1398` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.971533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffd67b-ceb0-4d6f-84d3-c03d94238935 None admin] RBAC: Authorizing `identity:get_user(user_id=3431213ea7d74427b80e1ddd6fdf1398)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:05.973693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffd67b-ceb0-4d6f-84d3-c03d94238935 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:05.975911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 166/329] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:50:05 2022] GET /identity/v3/users/3431213ea7d74427b80e1ddd6fdf1398 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:05.982283 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffd67b-ceb0-4d6f-84d3-c03d94238935 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:05.986005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:05.987559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787edae3-8ee6-45b3-a0ac-ac963563a64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:05.988259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787edae3-8ee6-45b3-a0ac-ac963563a64c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.988607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787edae3-8ee6-45b3-a0ac-ac963563a64c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.990113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787edae3-8ee6-45b3-a0ac-ac963563a64c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:06.001913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787edae3-8ee6-45b3-a0ac-ac963563a64c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-787edae3-8ee6-45b3-a0ac-ac963563a64c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:06.008388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:06.009285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:06.010354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 164/330] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:06.015509 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787edae3-8ee6-45b3-a0ac-ac963563a64c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:06.044685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:06.112826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffd67b-ceb0-4d6f-84d3-c03d94238935 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:06.143777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787edae3-8ee6-45b3-a0ac-ac963563a64c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:06.153215 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc33d50-9637-4f02-9cc9-f58d50d4d24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:06.153878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc33d50-9637-4f02-9cc9-f58d50d4d24b None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:06.154166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc33d50-9637-4f02-9cc9-f58d50d4d24b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:06.154446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc33d50-9637-4f02-9cc9-f58d50d4d24b None admin] PATH_INFO: `/v3/system/users/3431213ea7d74427b80e1ddd6fdf1398/roles/7fcd054cc7c549b18e4ac97d4a405e8c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:06.156541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc33d50-9637-4f02-9cc9-f58d50d4d24b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3431213ea7d74427b80e1ddd6fdf1398, role_id=7fcd054cc7c549b18e4ac97d4a405e8c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:06.157848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc33d50-9637-4f02-9cc9-f58d50d4d24b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:06.172268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 167/331] 10.222.0.86 () {58 vars in 1334 bytes} [Wed Aug 24 09:50:05 2022] PUT /identity/v3/system/users/3431213ea7d74427b80e1ddd6fdf1398/roles/7fcd054cc7c549b18e4ac97d4a405e8c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:50:06.190928 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33f6d-4aec-4a47-bea1-39f772e96ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:06.192918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33f6d-4aec-4a47-bea1-39f772e96ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:06.192918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33f6d-4aec-4a47-bea1-39f772e96ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:06.192918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33f6d-4aec-4a47-bea1-39f772e96ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:06.193375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d33f6d-4aec-4a47-bea1-39f772e96ea2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:06.193683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d33f6d-4aec-4a47-bea1-39f772e96ea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:06.195148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d33f6d-4aec-4a47-bea1-39f772e96ea2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:06.207791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 165/332] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:06.213386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc33d50-9637-4f02-9cc9-f58d50d4d24b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:06.293534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:06.335441 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc33d50-9637-4f02-9cc9-f58d50d4d24b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:06.377257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac28d6a-b89d-4f7c-b376-3d9023e1c137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:06.378300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac28d6a-b89d-4f7c-b376-3d9023e1c137 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:06.378831 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac28d6a-b89d-4f7c-b376-3d9023e1c137 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:06.379609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac28d6a-b89d-4f7c-b376-3d9023e1c137 None admin] PATH_INFO: `/v3/users/75c1fdec00b64c569072ce5d258cc6e6` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:06.380801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac28d6a-b89d-4f7c-b376-3d9023e1c137 None admin] RBAC: Authorizing `identity:get_user(user_id=75c1fdec00b64c569072ce5d258cc6e6)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:06.382251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac28d6a-b89d-4f7c-b376-3d9023e1c137 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:06.384115 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 168/333] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:50:06 2022] GET /identity/v3/users/75c1fdec00b64c569072ce5d258cc6e6 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:06.389911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33f6d-4aec-4a47-bea1-39f772e96ea2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:06.491126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:06.508874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33f6d-4aec-4a47-bea1-39f772e96ea2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:06.550317 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b642573-f7d3-499a-802e-3a4fd1209914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:06.551564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b642573-f7d3-499a-802e-3a4fd1209914 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:06.551841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b642573-f7d3-499a-802e-3a4fd1209914 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:06.552138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b642573-f7d3-499a-802e-3a4fd1209914 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:06.552860 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b642573-f7d3-499a-802e-3a4fd1209914 None admin] RBAC: Adding query filter params (role.id=7ab2687826544b8e80c709d6061162d4, user.id=75c1fdec00b64c569072ce5d258cc6e6, scope.system=all) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:06.553165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b642573-f7d3-499a-802e-3a4fd1209914 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:06.554603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b642573-f7d3-499a-802e-3a4fd1209914 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:06.569599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 166/334] 10.222.0.86 () {56 vars in 1308 bytes} [Wed Aug 24 09:50:06 2022] GET /identity/v3/role_assignments?user.id=75c1fdec00b64c569072ce5d258cc6e6&scope.system=all&role.id=7ab2687826544b8e80c709d6061162d4 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:07.925600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c92c71-c1a2-4934-928e-983e2c5737fe None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:07.926728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c92c71-c1a2-4934-928e-983e2c5737fe None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:07.926807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c92c71-c1a2-4934-928e-983e2c5737fe None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:07.928464 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 169/335] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:07.933459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d894ab4-6ed4-4fb5-b4a6-5fdcfb99f977 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:07.933877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d894ab4-6ed4-4fb5-b4a6-5fdcfb99f977 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:07.934047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d894ab4-6ed4-4fb5-b4a6-5fdcfb99f977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:07.957497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3d894ab4-6ed4-4fb5-b4a6-5fdcfb99f977 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:07.958802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3d894ab4-6ed4-4fb5-b4a6-5fdcfb99f977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:08.054101 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 167/336] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:08.173680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1258e-b8e4-46ec-9b87-8d4172d71f3f None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:08.173994 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1258e-b8e4-46ec-9b87-8d4172d71f3f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:08.174227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1258e-b8e4-46ec-9b87-8d4172d71f3f None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:08.175854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 170/337] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:08.183493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab2cc7-3e8d-4fc7-ab70-6d52a0838659 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:08.183783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab2cc7-3e8d-4fc7-ab70-6d52a0838659 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:08.184091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab2cc7-3e8d-4fc7-ab70-6d52a0838659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:08.207493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-28ab2cc7-3e8d-4fc7-ab70-6d52a0838659 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:08.208739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-28ab2cc7-3e8d-4fc7-ab70-6d52a0838659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:08.295301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 168/338] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:08.512835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed900ed-9760-4753-956d-98191b89d70c None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:08.512835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed900ed-9760-4753-956d-98191b89d70c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:08.512835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed900ed-9760-4753-956d-98191b89d70c None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:08.513625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 171/339] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:08.518050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459548d0-ac60-490d-8826-f057ae04832d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:08.518339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459548d0-ac60-490d-8826-f057ae04832d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:08.518616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459548d0-ac60-490d-8826-f057ae04832d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:08.529086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:08.545750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-459548d0-ac60-490d-8826-f057ae04832d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:08.546959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-459548d0-ac60-490d-8826-f057ae04832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:08.580239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed900ed-9760-4753-956d-98191b89d70c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:08.706060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 169/340] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:08.722295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed900ed-9760-4753-956d-98191b89d70c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:08.758919 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae798d-5fa8-4b11-97ff-2531aad0f22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:08.759908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae798d-5fa8-4b11-97ff-2531aad0f22f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:08.760409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae798d-5fa8-4b11-97ff-2531aad0f22f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:08.760874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae798d-5fa8-4b11-97ff-2531aad0f22f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:08.763990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:08.776547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae798d-5fa8-4b11-97ff-2531aad0f22f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a6ae798d-5fa8-4b11-97ff-2531aad0f22f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:08.781841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:08.782961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:08.783960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:08.794177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 172/341] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:08.799449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459548d0-ac60-490d-8826-f057ae04832d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:08.806337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae798d-5fa8-4b11-97ff-2531aad0f22f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:08.925620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae798d-5fa8-4b11-97ff-2531aad0f22f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:08.967962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb868bd0-710d-4830-a813-021f4d58a94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:08.968679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb868bd0-710d-4830-a813-021f4d58a94f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:08.968956 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb868bd0-710d-4830-a813-021f4d58a94f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:08.969228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb868bd0-710d-4830-a813-021f4d58a94f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:08.979876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb868bd0-710d-4830-a813-021f4d58a94f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-eb868bd0-710d-4830-a813-021f4d58a94f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:08.985981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:08.986900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:08.987691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 173/342] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:50:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:08.997247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb868bd0-710d-4830-a813-021f4d58a94f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.002152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459548d0-ac60-490d-8826-f057ae04832d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.081428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5a529-efaa-4e3d-9e18-d15ed1fa6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.081972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5a529-efaa-4e3d-9e18-d15ed1fa6c3d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.082287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5a529-efaa-4e3d-9e18-d15ed1fa6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.082567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5a529-efaa-4e3d-9e18-d15ed1fa6c3d None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.083251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5a529-efaa-4e3d-9e18-d15ed1fa6c3d None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:09.083557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5a529-efaa-4e3d-9e18-d15ed1fa6c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:09.084968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5a529-efaa-4e3d-9e18-d15ed1fa6c3d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:09.085637 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:09.087666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:09.103513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 170/343] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:09.104414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5a529-efaa-4e3d-9e18-d15ed1fa6c3d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.141880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb868bd0-710d-4830-a813-021f4d58a94f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.180310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e75090-2b67-4674-a429-7800d6334b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.180925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e75090-2b67-4674-a429-7800d6334b1e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.181206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e75090-2b67-4674-a429-7800d6334b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.181478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e75090-2b67-4674-a429-7800d6334b1e None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.182149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e75090-2b67-4674-a429-7800d6334b1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:09.182451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e75090-2b67-4674-a429-7800d6334b1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:09.183913 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e75090-2b67-4674-a429-7800d6334b1e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:09.192820 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 174/344] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:09.193652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e75090-2b67-4674-a429-7800d6334b1e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.293455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5a529-efaa-4e3d-9e18-d15ed1fa6c3d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.356660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e75090-2b67-4674-a429-7800d6334b1e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.368346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a619a-ec48-45f6-9fa2-ad588cba6e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.368988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a619a-ec48-45f6-9fa2-ad588cba6e2b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.369261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a619a-ec48-45f6-9fa2-ad588cba6e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.369530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a619a-ec48-45f6-9fa2-ad588cba6e2b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.390193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a619a-ec48-45f6-9fa2-ad588cba6e2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:09.397084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-330a619a-ec48-45f6-9fa2-ad588cba6e2b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:09.402431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:09.404621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:50:09.406575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:09.412139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 171/345] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:09.412139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a619a-ec48-45f6-9fa2-ad588cba6e2b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.429074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881141b3-3a18-4fa2-a5f3-00ce234a34cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.429691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881141b3-3a18-4fa2-a5f3-00ce234a34cc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.429971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881141b3-3a18-4fa2-a5f3-00ce234a34cc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.430245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881141b3-3a18-4fa2-a5f3-00ce234a34cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.499731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:09.524552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881141b3-3a18-4fa2-a5f3-00ce234a34cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:09.526776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881141b3-3a18-4fa2-a5f3-00ce234a34cc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:09.562009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a619a-ec48-45f6-9fa2-ad588cba6e2b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.605468 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a91fc-f863-4509-a4d7-058454cc5315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.606255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a91fc-f863-4509-a4d7-058454cc5315 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.606647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a91fc-f863-4509-a4d7-058454cc5315 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.607173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a91fc-f863-4509-a4d7-058454cc5315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.667834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 175/346] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:09.673073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881141b3-3a18-4fa2-a5f3-00ce234a34cc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.694001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a91fc-f863-4509-a4d7-058454cc5315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:09.694191 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:09.695479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a91fc-f863-4509-a4d7-058454cc5315 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:09.754092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:09.783250 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881141b3-3a18-4fa2-a5f3-00ce234a34cc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.809891 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 172/347] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:09.811315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb7255-992b-4c5e-9ef7-e1a8e84b1e35 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.811604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb7255-992b-4c5e-9ef7-e1a8e84b1e35 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.811874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb7255-992b-4c5e-9ef7-e1a8e84b1e35 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.813166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 173/348] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:09.814354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb7255-992b-4c5e-9ef7-e1a8e84b1e35 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.853250 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6a4ad-913c-41e6-8a7d-0ff94ef648d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.853974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6a4ad-913c-41e6-8a7d-0ff94ef648d1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.854320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6a4ad-913c-41e6-8a7d-0ff94ef648d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.854663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6a4ad-913c-41e6-8a7d-0ff94ef648d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.855405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6a4ad-913c-41e6-8a7d-0ff94ef648d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:09.855777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6a4ad-913c-41e6-8a7d-0ff94ef648d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:09.857609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6a4ad-913c-41e6-8a7d-0ff94ef648d1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:09.869993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 176/349] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:09.870674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6a4ad-913c-41e6-8a7d-0ff94ef648d1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.925328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb7255-992b-4c5e-9ef7-e1a8e84b1e35 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.967250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709110a2-b5d2-4f5e-b606-45ecdab0bb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.967868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709110a2-b5d2-4f5e-b606-45ecdab0bb31 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.968174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709110a2-b5d2-4f5e-b606-45ecdab0bb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.968550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709110a2-b5d2-4f5e-b606-45ecdab0bb31 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.969739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709110a2-b5d2-4f5e-b606-45ecdab0bb31 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:09.971090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:09.972407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709110a2-b5d2-4f5e-b606-45ecdab0bb31 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:09.974599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 174/350] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:09 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:09.976581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8794b5-31ca-40b0-8bfe-e5e0d876f5c1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.976864 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8794b5-31ca-40b0-8bfe-e5e0d876f5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.977138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8794b5-31ca-40b0-8bfe-e5e0d876f5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.997622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4f8794b5-31ca-40b0-8bfe-e5e0d876f5c1 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:09.998441 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6a4ad-913c-41e6-8a7d-0ff94ef648d1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.998816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4f8794b5-31ca-40b0-8bfe-e5e0d876f5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:10.019100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:10.045428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d042aa-9e01-4bf6-a328-827919baf304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.046069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d042aa-9e01-4bf6-a328-827919baf304 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.046350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d042aa-9e01-4bf6-a328-827919baf304 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.046624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d042aa-9e01-4bf6-a328-827919baf304 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.047777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d042aa-9e01-4bf6-a328-827919baf304 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:10.049858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d042aa-9e01-4bf6-a328-827919baf304 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:10.052089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 177/351] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:09 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:10.053339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d042aa-9e01-4bf6-a328-827919baf304 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:10.083628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 175/352] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:10.084475 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8794b5-31ca-40b0-8bfe-e5e0d876f5c1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:10.180115 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d042aa-9e01-4bf6-a328-827919baf304 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:10.205872 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8794b5-31ca-40b0-8bfe-e5e0d876f5c1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:10.232278 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f8737-4b73-40b2-b92d-cba65204385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.232922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f8737-4b73-40b2-b92d-cba65204385e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.233201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f8737-4b73-40b2-b92d-cba65204385e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.233476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f8737-4b73-40b2-b92d-cba65204385e None admin] PATH_INFO: `/v3/users/3431213ea7d74427b80e1ddd6fdf1398` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.234808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f8737-4b73-40b2-b92d-cba65204385e None admin] RBAC: Authorizing `identity:get_user(user_id=3431213ea7d74427b80e1ddd6fdf1398)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:10.236394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f8737-4b73-40b2-b92d-cba65204385e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:10.238170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 178/353] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:50:10 2022] GET /identity/v3/users/3431213ea7d74427b80e1ddd6fdf1398 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:10.239637 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f8737-4b73-40b2-b92d-cba65204385e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:10.247185 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b47ab8-a9f7-435a-84be-62085d04de32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.247820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b47ab8-a9f7-435a-84be-62085d04de32 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.248145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b47ab8-a9f7-435a-84be-62085d04de32 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.248424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b47ab8-a9f7-435a-84be-62085d04de32 None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.255217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b47ab8-a9f7-435a-84be-62085d04de32 None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:10.256938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b47ab8-a9f7-435a-84be-62085d04de32 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:10.258716 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 176/354] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:10 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:10.259071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:10.261812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde0825-ddb1-4307-b019-1e2fe4a4a37d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.262094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde0825-ddb1-4307-b019-1e2fe4a4a37d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.262441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde0825-ddb1-4307-b019-1e2fe4a4a37d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.263978 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 177/355] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:10.265709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde0825-ddb1-4307-b019-1e2fe4a4a37d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:10.287314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:10.352370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f8737-4b73-40b2-b92d-cba65204385e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:10.386898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde0825-ddb1-4307-b019-1e2fe4a4a37d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:10.387273 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd72486-98a6-4c83-85bb-a3004168ae24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.387923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd72486-98a6-4c83-85bb-a3004168ae24 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.388234 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd72486-98a6-4c83-85bb-a3004168ae24 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.388510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd72486-98a6-4c83-85bb-a3004168ae24 None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.389657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd72486-98a6-4c83-85bb-a3004168ae24 None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:10.391114 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd72486-98a6-4c83-85bb-a3004168ae24 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:10.392740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 179/356] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:10 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:10.393590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd72486-98a6-4c83-85bb-a3004168ae24 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:10.427153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8993c-4bd1-4753-a304-16f779b9ef1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.427816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8993c-4bd1-4753-a304-16f779b9ef1c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.428129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8993c-4bd1-4753-a304-16f779b9ef1c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.428504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8993c-4bd1-4753-a304-16f779b9ef1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.429281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8993c-4bd1-4753-a304-16f779b9ef1c None admin] RBAC: Adding query filter params (role.id=7fcd054cc7c549b18e4ac97d4a405e8c, user.id=3431213ea7d74427b80e1ddd6fdf1398, scope.system=all) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:10.429598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8993c-4bd1-4753-a304-16f779b9ef1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:10.431108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8993c-4bd1-4753-a304-16f779b9ef1c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:10.445140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 178/357] 10.222.0.86 () {56 vars in 1308 bytes} [Wed Aug 24 09:50:10 2022] GET /identity/v3/role_assignments?user.id=3431213ea7d74427b80e1ddd6fdf1398&scope.system=all&role.id=7fcd054cc7c549b18e4ac97d4a405e8c => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:10.447240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415f183-1968-4b49-b78c-0e9955bed4e1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.447240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415f183-1968-4b49-b78c-0e9955bed4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.447240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415f183-1968-4b49-b78c-0e9955bed4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.466895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6415f183-1968-4b49-b78c-0e9955bed4e1 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:10.468216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6415f183-1968-4b49-b78c-0e9955bed4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:10.486857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:10.516413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd72486-98a6-4c83-85bb-a3004168ae24 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:10.564483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17620b28-f981-4486-8ad7-9572303ff097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.564914 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620b28-f981-4486-8ad7-9572303ff097 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.565196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620b28-f981-4486-8ad7-9572303ff097 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.565467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620b28-f981-4486-8ad7-9572303ff097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.566216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17620b28-f981-4486-8ad7-9572303ff097 None admin] RBAC: Adding query filter params (group.id=29e80051419f49bc97ff5afbea57843b, role.id=7ab2687826544b8e80c709d6061162d4, scope.project.id=2ec79012680c43e3bfc414986fbb0b79) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:10.566554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17620b28-f981-4486-8ad7-9572303ff097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:10.568203 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17620b28-f981-4486-8ad7-9572303ff097 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:10.570795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 179/358] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:10.572550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415f183-1968-4b49-b78c-0e9955bed4e1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:10.575308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 180/359] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Aug 24 09:50:10 2022] GET /identity/v3/role_assignments?group.id=29e80051419f49bc97ff5afbea57843b&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=7ab2687826544b8e80c709d6061162d4 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:10.576481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17620b28-f981-4486-8ad7-9572303ff097 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:10.584664 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:10.732670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415f183-1968-4b49-b78c-0e9955bed4e1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:10.736973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17620b28-f981-4486-8ad7-9572303ff097 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:10.791925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5c8fd-cc7a-44af-990b-77488a625750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.794531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5c8fd-cc7a-44af-990b-77488a625750 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.795045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5c8fd-cc7a-44af-990b-77488a625750 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.795492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5c8fd-cc7a-44af-990b-77488a625750 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.795602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59401c2-a769-4c95-9ba7-0ad8cb02d159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.796379 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59401c2-a769-4c95-9ba7-0ad8cb02d159 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.796732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59401c2-a769-4c95-9ba7-0ad8cb02d159 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.797078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59401c2-a769-4c95-9ba7-0ad8cb02d159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.797892 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59401c2-a769-4c95-9ba7-0ad8cb02d159 None admin] RBAC: Adding query filter params (group.id=29e80051419f49bc97ff5afbea57843b, role.id=fc1186ecd0f74b2fad8155d528f268f9, scope.project.id=2ec79012680c43e3bfc414986fbb0b79) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:10.798277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59401c2-a769-4c95-9ba7-0ad8cb02d159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:10.800732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59401c2-a769-4c95-9ba7-0ad8cb02d159 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:10.812671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 180/360] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Aug 24 09:50:10 2022] GET /identity/v3/role_assignments?group.id=29e80051419f49bc97ff5afbea57843b&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=fc1186ecd0f74b2fad8155d528f268f9 => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:10.813897 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5c8fd-cc7a-44af-990b-77488a625750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:10.814317 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b76c92-b53b-4f4f-a34c-83f6315b04af None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.814603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b76c92-b53b-4f4f-a34c-83f6315b04af None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.814880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b76c92-b53b-4f4f-a34c-83f6315b04af None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.816221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 181/361] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-1ce5c8fd-cc7a-44af-990b-77488a625750 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:10.821828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:10.822751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:10.823660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:10.824308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5216d-a3e6-40df-a063-13c1c58638ee None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.824308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5216d-a3e6-40df-a063-13c1c58638ee None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.824308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5216d-a3e6-40df-a063-13c1c58638ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.826838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 181/362] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:10.832274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5c8fd-cc7a-44af-990b-77488a625750 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:10.847558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:10.851450 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-05c5216d-a3e6-40df-a063-13c1c58638ee None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:10.852745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-05c5216d-a3e6-40df-a063-13c1c58638ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:10.935577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:10.943285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 182/363] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:10.978586 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5216d-a3e6-40df-a063-13c1c58638ee None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:11.051838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5c8fd-cc7a-44af-990b-77488a625750 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:11.104908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a290f-8b35-4b34-9abe-f2c31eca78f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:11.105521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a290f-8b35-4b34-9abe-f2c31eca78f5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:11.105803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a290f-8b35-4b34-9abe-f2c31eca78f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:11.106075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a290f-8b35-4b34-9abe-f2c31eca78f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:11.106825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a290f-8b35-4b34-9abe-f2c31eca78f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:11.107132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a290f-8b35-4b34-9abe-f2c31eca78f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:11.108576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a290f-8b35-4b34-9abe-f2c31eca78f5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:11.122086 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 182/364] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:11.130180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a290f-8b35-4b34-9abe-f2c31eca78f5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:11.166989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5216d-a3e6-40df-a063-13c1c58638ee None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:11.226410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3b041-3657-4a02-811f-a053167deef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:11.227054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3b041-3657-4a02-811f-a053167deef4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:11.227333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3b041-3657-4a02-811f-a053167deef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:11.227606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3b041-3657-4a02-811f-a053167deef4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:11.240034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3b041-3657-4a02-811f-a053167deef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-20b3b041-3657-4a02-811f-a053167deef4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:11.245940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:11.246846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:11.248135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 183/365] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:50:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:11.255181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3b041-3657-4a02-811f-a053167deef4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:11.279927 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:11.323945 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a290f-8b35-4b34-9abe-f2c31eca78f5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:11.347415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:11.366609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4846a0-33af-481d-ab96-58ce122508d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:11.367221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4846a0-33af-481d-ab96-58ce122508d6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:11.367502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4846a0-33af-481d-ab96-58ce122508d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:11.367774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4846a0-33af-481d-ab96-58ce122508d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:11.395888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3b041-3657-4a02-811f-a053167deef4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:11.433754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c664b-deb9-4768-84b7-7e7cfe7f5cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:11.434455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c664b-deb9-4768-84b7-7e7cfe7f5cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:11.434741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c664b-deb9-4768-84b7-7e7cfe7f5cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:11.435020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c664b-deb9-4768-84b7-7e7cfe7f5cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:11.435775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c664b-deb9-4768-84b7-7e7cfe7f5cd8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:11.436096 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c664b-deb9-4768-84b7-7e7cfe7f5cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:11.437476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c664b-deb9-4768-84b7-7e7cfe7f5cd8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:11.437582 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4846a0-33af-481d-ab96-58ce122508d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:11.439012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4846a0-33af-481d-ab96-58ce122508d6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:11.446932 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 184/366] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:11.447641 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c664b-deb9-4768-84b7-7e7cfe7f5cd8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:11.511741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:11.546827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 183/367] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:11.553379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4846a0-33af-481d-ab96-58ce122508d6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:11.568203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c664b-deb9-4768-84b7-7e7cfe7f5cd8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:11.593723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:11.607936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a16b86-5117-468b-bae2-f9fddb557812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:11.608599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a16b86-5117-468b-bae2-f9fddb557812 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:11.608875 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a16b86-5117-468b-bae2-f9fddb557812 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:11.609146 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a16b86-5117-468b-bae2-f9fddb557812 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:11.621056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a16b86-5117-468b-bae2-f9fddb557812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-90a16b86-5117-468b-bae2-f9fddb557812 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:11.626784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:11.627682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 185/368] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:11.629549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a16b86-5117-468b-bae2-f9fddb557812 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:11.667203 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4846a0-33af-481d-ab96-58ce122508d6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:11.703650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cf599-03bf-48f7-95f4-031980f64ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:11.704289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cf599-03bf-48f7-95f4-031980f64ded None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:11.704568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cf599-03bf-48f7-95f4-031980f64ded None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:11.704843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cf599-03bf-48f7-95f4-031980f64ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:11.741088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a16b86-5117-468b-bae2-f9fddb557812 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:11.746541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:11.780891 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6214254-1150-468a-95c5-1ac530d19037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:11.781365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cf599-03bf-48f7-95f4-031980f64ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:11.781559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6214254-1150-468a-95c5-1ac530d19037 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:11.781781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6214254-1150-468a-95c5-1ac530d19037 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:11.782055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6214254-1150-468a-95c5-1ac530d19037 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:11.783280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6214254-1150-468a-95c5-1ac530d19037 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:11.785085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6214254-1150-468a-95c5-1ac530d19037 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:11.787705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 186/369] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:11 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:11.788509 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6214254-1150-468a-95c5-1ac530d19037 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:11.792812 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cf599-03bf-48f7-95f4-031980f64ded None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:11.847053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:11.899389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 184/370] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:11.900279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cf599-03bf-48f7-95f4-031980f64ded None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:11.907891 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6214254-1150-468a-95c5-1ac530d19037 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:11.945898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a869f4-45cb-450b-8a0b-fb9e923a7e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:11.946607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a869f4-45cb-450b-8a0b-fb9e923a7e48 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:11.946889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a869f4-45cb-450b-8a0b-fb9e923a7e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:11.947158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a869f4-45cb-450b-8a0b-fb9e923a7e48 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:11.947819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a869f4-45cb-450b-8a0b-fb9e923a7e48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:11.948161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a869f4-45cb-450b-8a0b-fb9e923a7e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:11.949659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a869f4-45cb-450b-8a0b-fb9e923a7e48 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:11.959021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 187/371] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:11.960007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a869f4-45cb-450b-8a0b-fb9e923a7e48 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:11.978304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:12.010130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cf599-03bf-48f7-95f4-031980f64ded None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:12.045818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784e588-7a35-4e40-9f7b-d168fad1111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:12.046455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784e588-7a35-4e40-9f7b-d168fad1111e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:12.046731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784e588-7a35-4e40-9f7b-d168fad1111e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:12.047005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784e588-7a35-4e40-9f7b-d168fad1111e None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:12.048384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784e588-7a35-4e40-9f7b-d168fad1111e None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:12.049844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784e588-7a35-4e40-9f7b-d168fad1111e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:12.051420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 185/372] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:11 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:12.052777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784e588-7a35-4e40-9f7b-d168fad1111e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:12.072015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a869f4-45cb-450b-8a0b-fb9e923a7e48 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:12.086583 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:12.112361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f283d-e30f-456a-a436-d54ca7193fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:12.113303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f283d-e30f-456a-a436-d54ca7193fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:12.113729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f283d-e30f-456a-a436-d54ca7193fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:12.114168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f283d-e30f-456a-a436-d54ca7193fb2 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:12.115529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f283d-e30f-456a-a436-d54ca7193fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:12.117249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f283d-e30f-456a-a436-d54ca7193fb2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:12.119240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 188/373] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:11 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:12.120189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f283d-e30f-456a-a436-d54ca7193fb2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:12.175111 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784e588-7a35-4e40-9f7b-d168fad1111e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:12.223108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fba61-8925-4998-be4a-b9ef0282304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:12.224910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fba61-8925-4998-be4a-b9ef0282304b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:12.225401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fba61-8925-4998-be4a-b9ef0282304b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:12.226034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fba61-8925-4998-be4a-b9ef0282304b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:12.237547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:12.247249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f283d-e30f-456a-a436-d54ca7193fb2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:12.289754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727f7c1-dbe9-4f87-8bfb-a21dab8bcd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:12.290364 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727f7c1-dbe9-4f87-8bfb-a21dab8bcd97 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:12.290914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727f7c1-dbe9-4f87-8bfb-a21dab8bcd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:12.291187 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727f7c1-dbe9-4f87-8bfb-a21dab8bcd97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:12.291916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727f7c1-dbe9-4f87-8bfb-a21dab8bcd97 None admin] RBAC: Adding query filter params (group.id=29e80051419f49bc97ff5afbea57843b, role.id=7ab2687826544b8e80c709d6061162d4, scope.project.id=10e7858e00034a8284453b08e8470748) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:12.292251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727f7c1-dbe9-4f87-8bfb-a21dab8bcd97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:12.293997 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727f7c1-dbe9-4f87-8bfb-a21dab8bcd97 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:12.301197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 189/374] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Aug 24 09:50:12 2022] GET /identity/v3/role_assignments?group.id=29e80051419f49bc97ff5afbea57843b&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=7ab2687826544b8e80c709d6061162d4 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:12.302070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727f7c1-dbe9-4f87-8bfb-a21dab8bcd97 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:12.355507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fba61-8925-4998-be4a-b9ef0282304b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:12.357038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fba61-8925-4998-be4a-b9ef0282304b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:12.364492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:12.429664 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727f7c1-dbe9-4f87-8bfb-a21dab8bcd97 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:12.471157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a64cec-89d8-4ba1-b202-6e9d202f53ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:12.471806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a64cec-89d8-4ba1-b202-6e9d202f53ee None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:12.472119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a64cec-89d8-4ba1-b202-6e9d202f53ee None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:12.472401 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a64cec-89d8-4ba1-b202-6e9d202f53ee None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:12.473618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a64cec-89d8-4ba1-b202-6e9d202f53ee None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:12.475337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a64cec-89d8-4ba1-b202-6e9d202f53ee None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:12.477994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 190/375] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:12 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:12.484034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a64cec-89d8-4ba1-b202-6e9d202f53ee None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:12.590851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 186/376] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:12.601123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fba61-8925-4998-be4a-b9ef0282304b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:12.626165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a64cec-89d8-4ba1-b202-6e9d202f53ee None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:12.647363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:12.687996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38fd59-e533-4a22-9c31-45e18a5d9062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:12.688640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38fd59-e533-4a22-9c31-45e18a5d9062 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:12.688971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38fd59-e533-4a22-9c31-45e18a5d9062 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:12.689251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38fd59-e533-4a22-9c31-45e18a5d9062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:12.689529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:12.690309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38fd59-e533-4a22-9c31-45e18a5d9062 None admin] RBAC: Adding query filter params (group.id=29e80051419f49bc97ff5afbea57843b, role.id=fc1186ecd0f74b2fad8155d528f268f9, scope.project.id=10e7858e00034a8284453b08e8470748) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:12.690309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38fd59-e533-4a22-9c31-45e18a5d9062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:12.691767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38fd59-e533-4a22-9c31-45e18a5d9062 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:12.698749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 191/377] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Aug 24 09:50:12 2022] GET /identity/v3/role_assignments?group.id=29e80051419f49bc97ff5afbea57843b&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=fc1186ecd0f74b2fad8155d528f268f9 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:12.732496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fba61-8925-4998-be4a-b9ef0282304b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:12.775690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feaf54b-9bd7-433a-89d9-a4ea3d58f49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:12.776369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feaf54b-9bd7-433a-89d9-a4ea3d58f49d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:12.776652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feaf54b-9bd7-433a-89d9-a4ea3d58f49d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:12.776924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feaf54b-9bd7-433a-89d9-a4ea3d58f49d None admin] PATH_INFO: `/v3/projects/f71a167399734714a60b777eea4e2ecc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:12.780806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feaf54b-9bd7-433a-89d9-a4ea3d58f49d None admin] RBAC: Authorizing `identity:get_project(project_id=f71a167399734714a60b777eea4e2ecc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:12.782275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feaf54b-9bd7-433a-89d9-a4ea3d58f49d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:12.784638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 187/378] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:12 2022] GET /identity/v3/projects/f71a167399734714a60b777eea4e2ecc => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:12.795973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38fd59-e533-4a22-9c31-45e18a5d9062 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:12.867503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c6062-a66f-4d42-bbb2-494a99d6cbc2 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:12.868088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c6062-a66f-4d42-bbb2-494a99d6cbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:12.868532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c6062-a66f-4d42-bbb2-494a99d6cbc2 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:12.870606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 188/379] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:12.882389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d57a23-aa6d-4e57-ab35-f0ff19106438 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:12.882792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d57a23-aa6d-4e57-ab35-f0ff19106438 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:12.883075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d57a23-aa6d-4e57-ab35-f0ff19106438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:12.929527 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-45d57a23-aa6d-4e57-ab35-f0ff19106438 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:12.931342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-45d57a23-aa6d-4e57-ab35-f0ff19106438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:12.951605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38fd59-e533-4a22-9c31-45e18a5d9062 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:13.001419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9afe7c-8660-4802-beec-659401f0a73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:13.002067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9afe7c-8660-4802-beec-659401f0a73b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:13.002350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9afe7c-8660-4802-beec-659401f0a73b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:13.002628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9afe7c-8660-4802-beec-659401f0a73b None admin] PATH_INFO: `/v3/groups/68e6cd5887ec4e018b0e8d72ef6d48b1` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:13.013080 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9afe7c-8660-4802-beec-659401f0a73b None admin] RBAC: Authorizing `identity:get_group(group_id=68e6cd5887ec4e018b0e8d72ef6d48b1)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:13.014567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9afe7c-8660-4802-beec-659401f0a73b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:13.020431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 192/380] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:12 2022] GET /identity/v3/groups/68e6cd5887ec4e018b0e8d72ef6d48b1 => generated 229 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:13.024066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9afe7c-8660-4802-beec-659401f0a73b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:13.046612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 189/381] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:13.083552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:13.150244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9afe7c-8660-4802-beec-659401f0a73b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:13.186672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c30f8-0ddb-43b3-90c9-18d846d662b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:13.187337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c30f8-0ddb-43b3-90c9-18d846d662b8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:13.187625 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c30f8-0ddb-43b3-90c9-18d846d662b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:13.187962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c30f8-0ddb-43b3-90c9-18d846d662b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:13.188730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c30f8-0ddb-43b3-90c9-18d846d662b8 None admin] RBAC: Adding query filter params (group.id=68e6cd5887ec4e018b0e8d72ef6d48b1, role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=f71a167399734714a60b777eea4e2ecc) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:13.189095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c30f8-0ddb-43b3-90c9-18d846d662b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:13.190622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c30f8-0ddb-43b3-90c9-18d846d662b8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:13.196949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 193/382] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Aug 24 09:50:13 2022] GET /identity/v3/role_assignments?group.id=68e6cd5887ec4e018b0e8d72ef6d48b1&scope.project.id=f71a167399734714a60b777eea4e2ecc&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:13.453204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d57a23-aa6d-4e57-ab35-f0ff19106438 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:13.492991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc52d8-fe22-458a-8267-d1e0128461b1 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:13.493493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:13.493577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc52d8-fe22-458a-8267-d1e0128461b1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:13.493577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc52d8-fe22-458a-8267-d1e0128461b1 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:13.494828 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 194/383] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:13.502520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de966-b8ae-4ca2-8848-2772c6226711 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:13.503107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de966-b8ae-4ca2-8848-2772c6226711 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:13.503588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de966-b8ae-4ca2-8848-2772c6226711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:13.528600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8b1de966-b8ae-4ca2-8848-2772c6226711 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:13.530770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8b1de966-b8ae-4ca2-8848-2772c6226711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:13.604194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d57a23-aa6d-4e57-ab35-f0ff19106438 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:13.676296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 195/384] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:13.706220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36c827-0f0f-4d39-bc56-4cc7262a1476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:13.706789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36c827-0f0f-4d39-bc56-4cc7262a1476 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:13.707067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36c827-0f0f-4d39-bc56-4cc7262a1476 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:13.707336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36c827-0f0f-4d39-bc56-4cc7262a1476 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:13.729199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36c827-0f0f-4d39-bc56-4cc7262a1476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-9d36c827-0f0f-4d39-bc56-4cc7262a1476 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:13.734137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:13.735172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:13.735999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 190/385] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:13.743228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1de966-b8ae-4ca2-8848-2772c6226711 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:13.804432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:13.893854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1de966-b8ae-4ca2-8848-2772c6226711 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:13.940914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc407a31-b1e6-41c9-bcaf-fe5786b15285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:13.941528 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc407a31-b1e6-41c9-bcaf-fe5786b15285 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:13.941810 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc407a31-b1e6-41c9-bcaf-fe5786b15285 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:13.942080 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc407a31-b1e6-41c9-bcaf-fe5786b15285 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:13.943281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc407a31-b1e6-41c9-bcaf-fe5786b15285 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:13.943586 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc407a31-b1e6-41c9-bcaf-fe5786b15285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:13.945022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc407a31-b1e6-41c9-bcaf-fe5786b15285 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:13.960546 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 196/386] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:13.966308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36c827-0f0f-4d39-bc56-4cc7262a1476 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:14.043168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc407a31-b1e6-41c9-bcaf-fe5786b15285 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:14.168406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:14.173484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36c827-0f0f-4d39-bc56-4cc7262a1476 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:14.230595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:14.241059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5756de-c093-4ad0-944e-2a0513e60007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:14.241700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5756de-c093-4ad0-944e-2a0513e60007 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.241981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5756de-c093-4ad0-944e-2a0513e60007 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.242258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5756de-c093-4ad0-944e-2a0513e60007 None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.243938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5756de-c093-4ad0-944e-2a0513e60007 None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:14.245969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5756de-c093-4ad0-944e-2a0513e60007 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:14.247730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 191/387] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:13 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:14.249732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc407a31-b1e6-41c9-bcaf-fe5786b15285 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:14.253560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5756de-c093-4ad0-944e-2a0513e60007 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:14.313335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35344f-bb81-4760-980e-cc5d240d147a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:14.313970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35344f-bb81-4760-980e-cc5d240d147a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.314248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35344f-bb81-4760-980e-cc5d240d147a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.314521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35344f-bb81-4760-980e-cc5d240d147a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.337617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35344f-bb81-4760-980e-cc5d240d147a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-1a35344f-bb81-4760-980e-cc5d240d147a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:14.344561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:14.345470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:14.346267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:14.348544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 197/388] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:50:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:14.353772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35344f-bb81-4760-980e-cc5d240d147a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:14.452102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5756de-c093-4ad0-944e-2a0513e60007 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:14.519932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9398b-ebc5-4c6e-aa54-890f8edeb2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:14.521886 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9398b-ebc5-4c6e-aa54-890f8edeb2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.525211 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9398b-ebc5-4c6e-aa54-890f8edeb2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.525702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9398b-ebc5-4c6e-aa54-890f8edeb2e6 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.528071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9398b-ebc5-4c6e-aa54-890f8edeb2e6 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:14.530608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9398b-ebc5-4c6e-aa54-890f8edeb2e6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:14.535753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 192/389] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:14 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:14.542300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9398b-ebc5-4c6e-aa54-890f8edeb2e6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:14.560202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35344f-bb81-4760-980e-cc5d240d147a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:14.624464 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:14.627494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1618b-e70b-433c-8bbb-9fd21892f9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:14.628155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1618b-e70b-433c-8bbb-9fd21892f9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.628553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1618b-e70b-433c-8bbb-9fd21892f9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.628840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1618b-e70b-433c-8bbb-9fd21892f9f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.629524 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1618b-e70b-433c-8bbb-9fd21892f9f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:14.629830 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1618b-e70b-433c-8bbb-9fd21892f9f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:14.631514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1618b-e70b-433c-8bbb-9fd21892f9f7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:14.641005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 198/390] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:14.649343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1618b-e70b-433c-8bbb-9fd21892f9f7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:14.689324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:14.744536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9398b-ebc5-4c6e-aa54-890f8edeb2e6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:14.813674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514d718-9b2d-4430-9691-dfd866f0fbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:14.814348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514d718-9b2d-4430-9691-dfd866f0fbd6 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.814806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514d718-9b2d-4430-9691-dfd866f0fbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.815246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514d718-9b2d-4430-9691-dfd866f0fbd6 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79/groups/29e80051419f49bc97ff5afbea57843b/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.818575 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514d718-9b2d-4430-9691-dfd866f0fbd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ec79012680c43e3bfc414986fbb0b79, group_id=29e80051419f49bc97ff5afbea57843b, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:14.820958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514d718-9b2d-4430-9691-dfd866f0fbd6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:14.829434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1618b-e70b-433c-8bbb-9fd21892f9f7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:14.839635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 193/391] 10.222.0.86 () {58 vars in 1442 bytes} [Wed Aug 24 09:50:14 2022] PUT /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79/groups/29e80051419f49bc97ff5afbea57843b/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:50:14.841297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ed06d-e96a-4c0d-8d75-f6f634a5ce47 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.841808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ed06d-e96a-4c0d-8d75-f6f634a5ce47 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.842284 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ed06d-e96a-4c0d-8d75-f6f634a5ce47 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.844151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 194/392] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:14.850742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96170d-3cee-48f3-896f-33112e0a4793 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.851398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96170d-3cee-48f3-896f-33112e0a4793 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.851887 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96170d-3cee-48f3-896f-33112e0a4793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.875382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5a96170d-3cee-48f3-896f-33112e0a4793 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:14.877281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5a96170d-3cee-48f3-896f-33112e0a4793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:14.895546 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a46f4a-1ed3-44e7-ace7-ca572dcf4435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:14.897193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a46f4a-1ed3-44e7-ace7-ca572dcf4435 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.897621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a46f4a-1ed3-44e7-ace7-ca572dcf4435 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.898054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a46f4a-1ed3-44e7-ace7-ca572dcf4435 None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.901212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a46f4a-1ed3-44e7-ace7-ca572dcf4435 None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:14.902983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a46f4a-1ed3-44e7-ace7-ca572dcf4435 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:14.905316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 199/393] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:14 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:14.911349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a46f4a-1ed3-44e7-ace7-ca572dcf4435 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:14.962310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 195/394] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:15.057606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:15.084927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a46f4a-1ed3-44e7-ace7-ca572dcf4435 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:15.130992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d4011-5583-4d82-9a5d-693266df5536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:15.131869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4011-5583-4d82-9a5d-693266df5536 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.132350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4011-5583-4d82-9a5d-693266df5536 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.132780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4011-5583-4d82-9a5d-693266df5536 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.136781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d4011-5583-4d82-9a5d-693266df5536 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:15.138433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d4011-5583-4d82-9a5d-693266df5536 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:15.140614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 200/395] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:14 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:15.146679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96170d-3cee-48f3-896f-33112e0a4793 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:15.177553 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:15.261651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96170d-3cee-48f3-896f-33112e0a4793 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:15.278514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc49b-d8aa-4d28-ab22-857f2a2eca44 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.279225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc49b-d8aa-4d28-ab22-857f2a2eca44 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.279730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc49b-d8aa-4d28-ab22-857f2a2eca44 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.282127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 201/396] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:15.286679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019813c-9c5e-4049-8f60-dc9d58f8a2ad None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.287385 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019813c-9c5e-4049-8f60-dc9d58f8a2ad None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.287877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019813c-9c5e-4049-8f60-dc9d58f8a2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.322887 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0fdd1-89cc-4c5e-acec-598df7ba4a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:15.323481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0fdd1-89cc-4c5e-acec-598df7ba4a7e None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.323763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0fdd1-89cc-4c5e-acec-598df7ba4a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.324054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0fdd1-89cc-4c5e-acec-598df7ba4a7e None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79/groups/29e80051419f49bc97ff5afbea57843b/roles/fc1186ecd0f74b2fad8155d528f268f9` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.325193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0019813c-9c5e-4049-8f60-dc9d58f8a2ad None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:15.326414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0019813c-9c5e-4049-8f60-dc9d58f8a2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:15.332838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0fdd1-89cc-4c5e-acec-598df7ba4a7e None admin] RBAC: Authorizing `identity:create_grant(project_id=2ec79012680c43e3bfc414986fbb0b79, group_id=29e80051419f49bc97ff5afbea57843b, role_id=fc1186ecd0f74b2fad8155d528f268f9)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:15.334344 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0fdd1-89cc-4c5e-acec-598df7ba4a7e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:15.365332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 196/397] 10.222.0.86 () {58 vars in 1442 bytes} [Wed Aug 24 09:50:15 2022] PUT /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79/groups/29e80051419f49bc97ff5afbea57843b/roles/fc1186ecd0f74b2fad8155d528f268f9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:50:15.366660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0fdd1-89cc-4c5e-acec-598df7ba4a7e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:15.417147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 202/398] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:15.500505 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:15.512466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0fdd1-89cc-4c5e-acec-598df7ba4a7e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:15.561109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af30446-2a55-47c8-800c-c9f233bd99d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:15.562556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af30446-2a55-47c8-800c-c9f233bd99d2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.563045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af30446-2a55-47c8-800c-c9f233bd99d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.563516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af30446-2a55-47c8-800c-c9f233bd99d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.576713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af30446-2a55-47c8-800c-c9f233bd99d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-5af30446-2a55-47c8-800c-c9f233bd99d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:15.582060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:15.583006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:15.583857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:15.587900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 197/399] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:15.593848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019813c-9c5e-4049-8f60-dc9d58f8a2ad None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:15.689165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:15.711342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019813c-9c5e-4049-8f60-dc9d58f8a2ad None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:15.752908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdc7d2-adce-4333-8d08-13b2f3476632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:15.753801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdc7d2-adce-4333-8d08-13b2f3476632 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.754236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdc7d2-adce-4333-8d08-13b2f3476632 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.754648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdc7d2-adce-4333-8d08-13b2f3476632 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.755491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdc7d2-adce-4333-8d08-13b2f3476632 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:15.755942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdc7d2-adce-4333-8d08-13b2f3476632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:15.757466 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdc7d2-adce-4333-8d08-13b2f3476632 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:15.760623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af30446-2a55-47c8-800c-c9f233bd99d2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:15.774478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 203/400] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:15.780374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdc7d2-adce-4333-8d08-13b2f3476632 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:15.916905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdc7d2-adce-4333-8d08-13b2f3476632 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:15.928690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af30446-2a55-47c8-800c-c9f233bd99d2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:15.956430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d84f9-ad94-4816-bde1-5c8ddebb2510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:15.957436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d84f9-ad94-4816-bde1-5c8ddebb2510 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.957925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d84f9-ad94-4816-bde1-5c8ddebb2510 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.958959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d84f9-ad94-4816-bde1-5c8ddebb2510 None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.960389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d84f9-ad94-4816-bde1-5c8ddebb2510 None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:15.962028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d84f9-ad94-4816-bde1-5c8ddebb2510 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:15.964119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 204/401] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:15 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:15.965450 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978c648-90a7-4908-875c-54b41eda0900 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.965932 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978c648-90a7-4908-875c-54b41eda0900 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.966584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978c648-90a7-4908-875c-54b41eda0900 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.968009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 205/402] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:15.973719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978c648-90a7-4908-875c-54b41eda0900 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:15.975594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359375c1-56df-42a3-b448-4acd0a8739cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:15.978172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359375c1-56df-42a3-b448-4acd0a8739cd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.978600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359375c1-56df-42a3-b448-4acd0a8739cd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.979029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359375c1-56df-42a3-b448-4acd0a8739cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.995987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:16.002033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359375c1-56df-42a3-b448-4acd0a8739cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-359375c1-56df-42a3-b448-4acd0a8739cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:16.007720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:16.008685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:16.009481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:16.012111 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 198/403] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:50:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:16.015754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec2b85-3a88-48db-bd92-e31542ca822d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:16.016411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec2b85-3a88-48db-bd92-e31542ca822d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:16.016902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec2b85-3a88-48db-bd92-e31542ca822d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:16.024382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:16.038567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5aec2b85-3a88-48db-bd92-e31542ca822d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:16.040213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5aec2b85-3a88-48db-bd92-e31542ca822d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:16.114523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978c648-90a7-4908-875c-54b41eda0900 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:16.129622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 199/404] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:16.132718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec2b85-3a88-48db-bd92-e31542ca822d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:16.158846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfe96e-f592-4062-b675-5ff72dd0915b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:16.159480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe96e-f592-4062-b675-5ff72dd0915b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:16.159762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe96e-f592-4062-b675-5ff72dd0915b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:16.160039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe96e-f592-4062-b675-5ff72dd0915b None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:16.161202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfe96e-f592-4062-b675-5ff72dd0915b None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:16.163830 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfe96e-f592-4062-b675-5ff72dd0915b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:16.166327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 206/405] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:15 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:16.172469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfe96e-f592-4062-b675-5ff72dd0915b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:16.255128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec2b85-3a88-48db-bd92-e31542ca822d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:16.263587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:16.291255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfe96e-f592-4062-b675-5ff72dd0915b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:16.300354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:16.307512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d714798-7732-4e54-a6eb-276e70709a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:16.308573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d714798-7732-4e54-a6eb-276e70709a8c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:16.309041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d714798-7732-4e54-a6eb-276e70709a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:16.309620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d714798-7732-4e54-a6eb-276e70709a8c None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:16.310639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d714798-7732-4e54-a6eb-276e70709a8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:16.311126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d714798-7732-4e54-a6eb-276e70709a8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:16.312807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d714798-7732-4e54-a6eb-276e70709a8c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:16.326402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 200/406] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:16.330232 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779448dc-1ceb-44d4-9fb6-841363fabfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:16.331477 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779448dc-1ceb-44d4-9fb6-841363fabfdc None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:16.331936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779448dc-1ceb-44d4-9fb6-841363fabfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:16.332341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d714798-7732-4e54-a6eb-276e70709a8c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:16.332621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779448dc-1ceb-44d4-9fb6-841363fabfdc None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748/groups/29e80051419f49bc97ff5afbea57843b/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:16.335551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779448dc-1ceb-44d4-9fb6-841363fabfdc None admin] RBAC: Authorizing `identity:create_grant(project_id=10e7858e00034a8284453b08e8470748, group_id=29e80051419f49bc97ff5afbea57843b, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:16.338548 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779448dc-1ceb-44d4-9fb6-841363fabfdc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:16.358837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 207/407] 10.222.0.86 () {58 vars in 1442 bytes} [Wed Aug 24 09:50:16 2022] PUT /identity/v3/projects/10e7858e00034a8284453b08e8470748/groups/29e80051419f49bc97ff5afbea57843b/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:50:16.504871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d714798-7732-4e54-a6eb-276e70709a8c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:16.548223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab91d05-b155-4638-88c4-ba8afc42b467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:16.548848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab91d05-b155-4638-88c4-ba8afc42b467 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:16.549121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab91d05-b155-4638-88c4-ba8afc42b467 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:16.549385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab91d05-b155-4638-88c4-ba8afc42b467 None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:16.551063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab91d05-b155-4638-88c4-ba8afc42b467 None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:16.553370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab91d05-b155-4638-88c4-ba8afc42b467 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:16.555276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 201/408] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:16 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:16.561221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779448dc-1ceb-44d4-9fb6-841363fabfdc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:16.600121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab91d05-b155-4638-88c4-ba8afc42b467 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:16.719615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779448dc-1ceb-44d4-9fb6-841363fabfdc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:16.724421 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:16.768232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab91d05-b155-4638-88c4-ba8afc42b467 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:16.792153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f898305-f40b-470e-b3ff-97711ecdbd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:16.793033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f898305-f40b-470e-b3ff-97711ecdbd05 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:16.793469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f898305-f40b-470e-b3ff-97711ecdbd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:16.793880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f898305-f40b-470e-b3ff-97711ecdbd05 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:16.795438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f898305-f40b-470e-b3ff-97711ecdbd05 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:16.798161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f898305-f40b-470e-b3ff-97711ecdbd05 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:16.800173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 208/409] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:16 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:16.809547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f898305-f40b-470e-b3ff-97711ecdbd05 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:16.827377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:16.830678 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc5342-dde5-4839-9d91-2a4f953469fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:16.831551 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc5342-dde5-4839-9d91-2a4f953469fd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:16.831977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc5342-dde5-4839-9d91-2a4f953469fd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:16.832472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc5342-dde5-4839-9d91-2a4f953469fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:16.868693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc5342-dde5-4839-9d91-2a4f953469fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-7abc5342-dde5-4839-9d91-2a4f953469fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:16.875038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:16.876184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:16.877201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:16.885850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 202/410] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:16.891326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc5342-dde5-4839-9d91-2a4f953469fd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:16.933184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f898305-f40b-470e-b3ff-97711ecdbd05 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:16.973240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37513c-fcdd-4887-a24d-fc1f5a3711d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:16.974003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37513c-fcdd-4887-a24d-fc1f5a3711d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:16.975234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37513c-fcdd-4887-a24d-fc1f5a3711d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:16.975609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37513c-fcdd-4887-a24d-fc1f5a3711d0 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748/groups/29e80051419f49bc97ff5afbea57843b/roles/fc1186ecd0f74b2fad8155d528f268f9` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:16.978220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37513c-fcdd-4887-a24d-fc1f5a3711d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=10e7858e00034a8284453b08e8470748, group_id=29e80051419f49bc97ff5afbea57843b, role_id=fc1186ecd0f74b2fad8155d528f268f9)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:16.979970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37513c-fcdd-4887-a24d-fc1f5a3711d0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:16.996839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 209/411] 10.222.0.86 () {58 vars in 1442 bytes} [Wed Aug 24 09:50:16 2022] PUT /identity/v3/projects/10e7858e00034a8284453b08e8470748/groups/29e80051419f49bc97ff5afbea57843b/roles/fc1186ecd0f74b2fad8155d528f268f9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:50:17.082506 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc5342-dde5-4839-9d91-2a4f953469fd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:17.108474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:17.124105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f647ed8-ee13-490b-ab24-48d4506ab278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:17.124715 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f647ed8-ee13-490b-ab24-48d4506ab278 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:17.124992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f647ed8-ee13-490b-ab24-48d4506ab278 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:17.125264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f647ed8-ee13-490b-ab24-48d4506ab278 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:17.125933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f647ed8-ee13-490b-ab24-48d4506ab278 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:17.126232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f647ed8-ee13-490b-ab24-48d4506ab278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:17.127681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f647ed8-ee13-490b-ab24-48d4506ab278 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:17.138929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 203/412] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:17.144396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37513c-fcdd-4887-a24d-fc1f5a3711d0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:17.240668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:17.273455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37513c-fcdd-4887-a24d-fc1f5a3711d0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:17.309125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0998e-2db1-4909-b9e0-15e0a215c9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:17.309768 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0998e-2db1-4909-b9e0-15e0a215c9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:17.310050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0998e-2db1-4909-b9e0-15e0a215c9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:17.310327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0998e-2db1-4909-b9e0-15e0a215c9b9 None admin] PATH_INFO: `/v3/groups/68e6cd5887ec4e018b0e8d72ef6d48b1` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:17.311455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0998e-2db1-4909-b9e0-15e0a215c9b9 None admin] RBAC: Authorizing `identity:get_group(group_id=68e6cd5887ec4e018b0e8d72ef6d48b1)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:17.312923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0998e-2db1-4909-b9e0-15e0a215c9b9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:17.314826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 210/413] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:17 2022] GET /identity/v3/groups/68e6cd5887ec4e018b0e8d72ef6d48b1 => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:17.320259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f647ed8-ee13-490b-ab24-48d4506ab278 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:17.458298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f647ed8-ee13-490b-ab24-48d4506ab278 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:17.505905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f429dc3-977d-44ba-aecb-8c469490814f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:17.506533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f429dc3-977d-44ba-aecb-8c469490814f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:17.506807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f429dc3-977d-44ba-aecb-8c469490814f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:17.507074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f429dc3-977d-44ba-aecb-8c469490814f None admin] PATH_INFO: `/v3/projects/f71a167399734714a60b777eea4e2ecc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:17.508227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f429dc3-977d-44ba-aecb-8c469490814f None admin] RBAC: Authorizing `identity:get_project(project_id=f71a167399734714a60b777eea4e2ecc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:17.509674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f429dc3-977d-44ba-aecb-8c469490814f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:17.511599 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 204/414] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:17 2022] GET /identity/v3/projects/f71a167399734714a60b777eea4e2ecc => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:17.517074 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0998e-2db1-4909-b9e0-15e0a215c9b9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:17.659825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0998e-2db1-4909-b9e0-15e0a215c9b9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:17.668386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648ad18-21f1-40df-b2cd-a2a2ed51e9cb None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:17.668907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648ad18-21f1-40df-b2cd-a2a2ed51e9cb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:17.668992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648ad18-21f1-40df-b2cd-a2a2ed51e9cb None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:17.670849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 205/415] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:50:17.676015 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb4efc-b776-452a-9589-dda48ff65d88 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:17.676322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb4efc-b776-452a-9589-dda48ff65d88 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:17.676594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb4efc-b776-452a-9589-dda48ff65d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:17.707350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8531811-5dc4-4e61-b705-8a6478567587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:17.708441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8531811-5dc4-4e61-b705-8a6478567587 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:17.709565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8531811-5dc4-4e61-b705-8a6478567587 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:17.710327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8531811-5dc4-4e61-b705-8a6478567587 None admin] PATH_INFO: `/v3/projects/f71a167399734714a60b777eea4e2ecc/groups/68e6cd5887ec4e018b0e8d72ef6d48b1/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:17.712482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8531811-5dc4-4e61-b705-8a6478567587 None admin] RBAC: Authorizing `identity:create_grant(project_id=f71a167399734714a60b777eea4e2ecc, group_id=68e6cd5887ec4e018b0e8d72ef6d48b1, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:17.714182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8531811-5dc4-4e61-b705-8a6478567587 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:17.719946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-31fb4efc-b776-452a-9589-dda48ff65d88 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:17.721295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-31fb4efc-b776-452a-9589-dda48ff65d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:17.724505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:17.732012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 211/416] 10.222.0.86 () {58 vars in 1442 bytes} [Wed Aug 24 09:50:17 2022] PUT /identity/v3/projects/f71a167399734714a60b777eea4e2ecc/groups/68e6cd5887ec4e018b0e8d72ef6d48b1/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:50:17.844324 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:17.874367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 206/417] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:18.145974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4689231-fbb4-4715-92f3-fdb8792fece0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:18.147181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4689231-fbb4-4715-92f3-fdb8792fece0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:18.147682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4689231-fbb4-4715-92f3-fdb8792fece0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:18.149572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 212/418] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:18.155223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54c0fc-521f-4e3d-b6c0-984ea708e13c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:18.155499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54c0fc-521f-4e3d-b6c0-984ea708e13c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:18.155775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54c0fc-521f-4e3d-b6c0-984ea708e13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:18.185058 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4d54c0fc-521f-4e3d-b6c0-984ea708e13c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:18.186243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4d54c0fc-521f-4e3d-b6c0-984ea708e13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:18.190304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4689231-fbb4-4715-92f3-fdb8792fece0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:18.318311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4689231-fbb4-4715-92f3-fdb8792fece0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:18.334284 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 207/419] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:18.368388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e04e26-a4fb-448b-97e9-e91dbaabb177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:18.376078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e04e26-a4fb-448b-97e9-e91dbaabb177 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:18.376668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e04e26-a4fb-448b-97e9-e91dbaabb177 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:18.377152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e04e26-a4fb-448b-97e9-e91dbaabb177 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:18.388900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e04e26-a4fb-448b-97e9-e91dbaabb177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-05e04e26-a4fb-448b-97e9-e91dbaabb177 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:18.398705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:18.399609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:18.400446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:18.403031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 213/420] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:18.408611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54c0fc-521f-4e3d-b6c0-984ea708e13c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:18.567376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54c0fc-521f-4e3d-b6c0-984ea708e13c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:18.590560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:18.622585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f3909-5462-4613-a103-564651f0deda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:18.623311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f3909-5462-4613-a103-564651f0deda None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:18.623734 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f3909-5462-4613-a103-564651f0deda None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:18.624145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f3909-5462-4613-a103-564651f0deda None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:18.624822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f3909-5462-4613-a103-564651f0deda None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:18.625129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f3909-5462-4613-a103-564651f0deda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:18.626517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f3909-5462-4613-a103-564651f0deda None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:18.639772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 208/421] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:18.645342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e04e26-a4fb-448b-97e9-e91dbaabb177 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:18.715805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f3909-5462-4613-a103-564651f0deda None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:18.740381 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:18.800007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e04e26-a4fb-448b-97e9-e91dbaabb177 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:18.838043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d5e43-abb8-4e7a-afde-7b42a44fca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:18.838649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d5e43-abb8-4e7a-afde-7b42a44fca5c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:18.838932 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d5e43-abb8-4e7a-afde-7b42a44fca5c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:18.839207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d5e43-abb8-4e7a-afde-7b42a44fca5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:18.885590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f3909-5462-4613-a103-564651f0deda None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:18.918346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d5e43-abb8-4e7a-afde-7b42a44fca5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:18.920097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d5e43-abb8-4e7a-afde-7b42a44fca5c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:18.921682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c828a6-71c8-4c43-aba6-40862b557206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:18.922695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c828a6-71c8-4c43-aba6-40862b557206 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:18.923038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c828a6-71c8-4c43-aba6-40862b557206 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:18.924391 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c828a6-71c8-4c43-aba6-40862b557206 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:18.942612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c828a6-71c8-4c43-aba6-40862b557206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-15c828a6-71c8-4c43-aba6-40862b557206 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:18.951144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:18.952412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:18.953206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:18.954725 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 209/422] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:50:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:18.954801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc62ed-5e83-4f8c-97eb-457d4c246918 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:18.955088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc62ed-5e83-4f8c-97eb-457d4c246918 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:18.955364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc62ed-5e83-4f8c-97eb-457d4c246918 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:18.961294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 210/423] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:18.962100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc62ed-5e83-4f8c-97eb-457d4c246918 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:18.994499 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.002254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.049173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 214/424] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:19.051296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b4b92-5973-4c05-8950-f7917cba37ef None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.052084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b4b92-5973-4c05-8950-f7917cba37ef None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.053361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b4b92-5973-4c05-8950-f7917cba37ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.074878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc62ed-5e83-4f8c-97eb-457d4c246918 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:19.076642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-496b4b92-5973-4c05-8950-f7917cba37ef None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:19.078912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-496b4b92-5973-4c05-8950-f7917cba37ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:19.114303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68612fe1-07a6-42b4-aff9-95569b8a7bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:19.114954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68612fe1-07a6-42b4-aff9-95569b8a7bdf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.115842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68612fe1-07a6-42b4-aff9-95569b8a7bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.116015 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68612fe1-07a6-42b4-aff9-95569b8a7bdf None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.116710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68612fe1-07a6-42b4-aff9-95569b8a7bdf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:19.117068 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68612fe1-07a6-42b4-aff9-95569b8a7bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:19.118485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68612fe1-07a6-42b4-aff9-95569b8a7bdf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:19.128673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 211/425] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:19.129116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68612fe1-07a6-42b4-aff9-95569b8a7bdf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:19.169056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 215/426] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:19.170700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b4b92-5973-4c05-8950-f7917cba37ef None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:19.281782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.290302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68612fe1-07a6-42b4-aff9-95569b8a7bdf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:19.308832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.323188 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b4b92-5973-4c05-8950-f7917cba37ef None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:19.335502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be312b31-67dc-4b17-a7c7-281e1e4d8bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:19.336478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be312b31-67dc-4b17-a7c7-281e1e4d8bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.336771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be312b31-67dc-4b17-a7c7-281e1e4d8bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.337057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be312b31-67dc-4b17-a7c7-281e1e4d8bd4 None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.338440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be312b31-67dc-4b17-a7c7-281e1e4d8bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:19.340102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be312b31-67dc-4b17-a7c7-281e1e4d8bd4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:19.342729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 212/427] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:19 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:19.344062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f275c7-76d2-4b72-8a84-2eaade3bf2ce None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.344342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f275c7-76d2-4b72-8a84-2eaade3bf2ce None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.344635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f275c7-76d2-4b72-8a84-2eaade3bf2ce None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.348175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 213/428] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:19.349229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f275c7-76d2-4b72-8a84-2eaade3bf2ce None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:19.366312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1b678-ca47-4959-b77b-4ff725be2b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:19.367319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1b678-ca47-4959-b77b-4ff725be2b76 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.367863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1b678-ca47-4959-b77b-4ff725be2b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.368354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1b678-ca47-4959-b77b-4ff725be2b76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.446501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1b678-ca47-4959-b77b-4ff725be2b76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:19.447983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1b678-ca47-4959-b77b-4ff725be2b76 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:19.480835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f275c7-76d2-4b72-8a84-2eaade3bf2ce None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:19.518150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366c4cb-7b16-4082-90e8-806df985a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:19.519492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366c4cb-7b16-4082-90e8-806df985a2ef None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.519989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366c4cb-7b16-4082-90e8-806df985a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.520674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366c4cb-7b16-4082-90e8-806df985a2ef None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.522769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366c4cb-7b16-4082-90e8-806df985a2ef None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:19.525194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366c4cb-7b16-4082-90e8-806df985a2ef None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:19.527298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 214/429] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:19 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:19.528680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f9137-4a90-4fb1-b048-a23518d42a4b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.529119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f9137-4a90-4fb1-b048-a23518d42a4b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.529550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f9137-4a90-4fb1-b048-a23518d42a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.550506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.551710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-407f9137-4a90-4fb1-b048-a23518d42a4b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:19.554459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-407f9137-4a90-4fb1-b048-a23518d42a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:19.569779 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 216/430] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:19.574559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1b678-ca47-4959-b77b-4ff725be2b76 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:19.594703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.642243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 215/431] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:19.643097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f9137-4a90-4fb1-b048-a23518d42a4b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:19.698465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1b678-ca47-4959-b77b-4ff725be2b76 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:19.744829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9143d-0180-43d8-87fc-92a87147270f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:19.745462 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9143d-0180-43d8-87fc-92a87147270f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.745719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9143d-0180-43d8-87fc-92a87147270f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.745996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9143d-0180-43d8-87fc-92a87147270f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.756459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f9137-4a90-4fb1-b048-a23518d42a4b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:19.761428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9143d-0180-43d8-87fc-92a87147270f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3dc9143d-0180-43d8-87fc-92a87147270f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:19.767087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:19.768009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 217/432] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:19.768843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9143d-0180-43d8-87fc-92a87147270f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:19.810836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612e2b4-aa28-44d9-b17b-7b593980b5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:19.812239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612e2b4-aa28-44d9-b17b-7b593980b5fd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.813196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612e2b4-aa28-44d9-b17b-7b593980b5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.813691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612e2b4-aa28-44d9-b17b-7b593980b5fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.815454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612e2b4-aa28-44d9-b17b-7b593980b5fd None admin] RBAC: Adding query filter params (group.id=29e80051419f49bc97ff5afbea57843b, role.id=7ab2687826544b8e80c709d6061162d4, scope.project.id=2ec79012680c43e3bfc414986fbb0b79) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:19.815930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612e2b4-aa28-44d9-b17b-7b593980b5fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:19.817563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612e2b4-aa28-44d9-b17b-7b593980b5fd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:19.825718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 216/433] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Aug 24 09:50:19 2022] GET /identity/v3/role_assignments?group.id=29e80051419f49bc97ff5afbea57843b&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=7ab2687826544b8e80c709d6061162d4 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:19.827003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612e2b4-aa28-44d9-b17b-7b593980b5fd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:19.833431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.893239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.966911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9143d-0180-43d8-87fc-92a87147270f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:19.998701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612e2b4-aa28-44d9-b17b-7b593980b5fd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.030629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dcca8-751a-4e5c-a065-9ec2648bd30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.031268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dcca8-751a-4e5c-a065-9ec2648bd30d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.031642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dcca8-751a-4e5c-a065-9ec2648bd30d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.031885 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dcca8-751a-4e5c-a065-9ec2648bd30d None admin] PATH_INFO: `/v3/projects/2ec79012680c43e3bfc414986fbb0b79` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.033179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dcca8-751a-4e5c-a065-9ec2648bd30d None admin] RBAC: Authorizing `identity:get_project(project_id=2ec79012680c43e3bfc414986fbb0b79)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.034796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dcca8-751a-4e5c-a065-9ec2648bd30d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.036591 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 218/434] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:19 2022] GET /identity/v3/projects/2ec79012680c43e3bfc414986fbb0b79 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:20.040334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dcca8-751a-4e5c-a065-9ec2648bd30d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.043406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f63b6-4298-4070-ad42-62fdd00fbea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.044106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f63b6-4298-4070-ad42-62fdd00fbea0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.044387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f63b6-4298-4070-ad42-62fdd00fbea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.044662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f63b6-4298-4070-ad42-62fdd00fbea0 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.046698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f63b6-4298-4070-ad42-62fdd00fbea0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:20.046698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f63b6-4298-4070-ad42-62fdd00fbea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.047800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f63b6-4298-4070-ad42-62fdd00fbea0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.058356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 217/435] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:50:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:20.059595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818aa371-7d08-4d06-88ef-9b6aa935ffe0 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.059879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818aa371-7d08-4d06-88ef-9b6aa935ffe0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.060181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818aa371-7d08-4d06-88ef-9b6aa935ffe0 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.061473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 218/436] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:20.062095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818aa371-7d08-4d06-88ef-9b6aa935ffe0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.160600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dcca8-751a-4e5c-a065-9ec2648bd30d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.168354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:20.184998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:20.189773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818aa371-7d08-4d06-88ef-9b6aa935ffe0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.207814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b13b63-4397-479c-b6d6-93bdc729e20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.208533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b13b63-4397-479c-b6d6-93bdc729e20a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.208813 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b13b63-4397-479c-b6d6-93bdc729e20a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.209118 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b13b63-4397-479c-b6d6-93bdc729e20a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.221941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b13b63-4397-479c-b6d6-93bdc729e20a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-67b13b63-4397-479c-b6d6-93bdc729e20a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:20.227616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:20.228739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 219/437] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:20.229723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b13b63-4397-479c-b6d6-93bdc729e20a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.232467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12eac-19df-4631-bbb6-152b968e9d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.232467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12eac-19df-4631-bbb6-152b968e9d11 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.232467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12eac-19df-4631-bbb6-152b968e9d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.232745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12eac-19df-4631-bbb6-152b968e9d11 None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.234795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12eac-19df-4631-bbb6-152b968e9d11 None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.235298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12eac-19df-4631-bbb6-152b968e9d11 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.237204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 219/438] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:20.238158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6549e3-bf97-44e7-a1f6-3a160b00d3af None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.238502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6549e3-bf97-44e7-a1f6-3a160b00d3af None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.238782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6549e3-bf97-44e7-a1f6-3a160b00d3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.258677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4d6549e3-bf97-44e7-a1f6-3a160b00d3af None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:20.259888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4d6549e3-bf97-44e7-a1f6-3a160b00d3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:20.332084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 220/439] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:20.332662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6549e3-bf97-44e7-a1f6-3a160b00d3af None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.347350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b13b63-4397-479c-b6d6-93bdc729e20a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.386324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6b272-d881-41d8-8e39-3166af440813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.386931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6b272-d881-41d8-8e39-3166af440813 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.387214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6b272-d881-41d8-8e39-3166af440813 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.387486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6b272-d881-41d8-8e39-3166af440813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.428475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:20.439442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:20.445614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6549e3-bf97-44e7-a1f6-3a160b00d3af None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.468154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6b272-d881-41d8-8e39-3166af440813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.470022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6b272-d881-41d8-8e39-3166af440813 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.482406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66abde0-a52c-45b4-8157-83736ed9e9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.483041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66abde0-a52c-45b4-8157-83736ed9e9ef None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.483295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66abde0-a52c-45b4-8157-83736ed9e9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.483588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66abde0-a52c-45b4-8157-83736ed9e9ef None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.484428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66abde0-a52c-45b4-8157-83736ed9e9ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:20.484666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66abde0-a52c-45b4-8157-83736ed9e9ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.485921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66abde0-a52c-45b4-8157-83736ed9e9ef None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.498865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 221/440] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:20.498865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66abde0-a52c-45b4-8157-83736ed9e9ef None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.591516 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 220/441] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:20.591516 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6b272-d881-41d8-8e39-3166af440813 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.608250 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66abde0-a52c-45b4-8157-83736ed9e9ef None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.641388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b03ee1-7ce2-46e2-b8b0-0d6d2bedf08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.641979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b03ee1-7ce2-46e2-b8b0-0d6d2bedf08d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.642259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b03ee1-7ce2-46e2-b8b0-0d6d2bedf08d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.642527 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b03ee1-7ce2-46e2-b8b0-0d6d2bedf08d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.643241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b03ee1-7ce2-46e2-b8b0-0d6d2bedf08d None admin] RBAC: Adding query filter params (group.id=29e80051419f49bc97ff5afbea57843b, role.id=fc1186ecd0f74b2fad8155d528f268f9, scope.project.id=2ec79012680c43e3bfc414986fbb0b79) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:20.643552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b03ee1-7ce2-46e2-b8b0-0d6d2bedf08d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.645040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b03ee1-7ce2-46e2-b8b0-0d6d2bedf08d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.651180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 222/442] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/role_assignments?group.id=29e80051419f49bc97ff5afbea57843b&scope.project.id=2ec79012680c43e3bfc414986fbb0b79&role.id=fc1186ecd0f74b2fad8155d528f268f9 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:20.651965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b03ee1-7ce2-46e2-b8b0-0d6d2bedf08d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.668896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:20.669858 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:20.700593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6b272-d881-41d8-8e39-3166af440813 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.741786 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6372b-0a40-41bb-b5f8-9fc9559600dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.742505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6372b-0a40-41bb-b5f8-9fc9559600dc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.742856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6372b-0a40-41bb-b5f8-9fc9559600dc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.743139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6372b-0a40-41bb-b5f8-9fc9559600dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.767321 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b03ee1-7ce2-46e2-b8b0-0d6d2bedf08d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.808252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c001b52-3088-4de0-b180-20c285d77bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.808882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c001b52-3088-4de0-b180-20c285d77bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.809166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c001b52-3088-4de0-b180-20c285d77bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.809436 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c001b52-3088-4de0-b180-20c285d77bc6 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.810602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c001b52-3088-4de0-b180-20c285d77bc6 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.812157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c001b52-3088-4de0-b180-20c285d77bc6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.813780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 223/443] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:20.814548 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c001b52-3088-4de0-b180-20c285d77bc6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.829676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa6372b-0a40-41bb-b5f8-9fc9559600dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.831489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa6372b-0a40-41bb-b5f8-9fc9559600dc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.918029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:20.924917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:20.941881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c001b52-3088-4de0-b180-20c285d77bc6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.954265 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 221/444] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:20.955205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6372b-0a40-41bb-b5f8-9fc9559600dc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.983966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc4a2c-c57f-4afe-b81b-5853b8391ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.984637 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc4a2c-c57f-4afe-b81b-5853b8391ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.984916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc4a2c-c57f-4afe-b81b-5853b8391ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.985189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc4a2c-c57f-4afe-b81b-5853b8391ba2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.996639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc4a2c-c57f-4afe-b81b-5853b8391ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-b7bc4a2c-c57f-4afe-b81b-5853b8391ba2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:21.002759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:21.003696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:21.004563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 224/445] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:21.005235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc4a2c-c57f-4afe-b81b-5853b8391ba2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:21.063091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6372b-0a40-41bb-b5f8-9fc9559600dc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:21.098124 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4436e-56ff-45a1-aff3-9aff4b677f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:21.098744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4436e-56ff-45a1-aff3-9aff4b677f44 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:21.099023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4436e-56ff-45a1-aff3-9aff4b677f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:21.099295 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4436e-56ff-45a1-aff3-9aff4b677f44 None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:21.100436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4436e-56ff-45a1-aff3-9aff4b677f44 None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:21.101866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4436e-56ff-45a1-aff3-9aff4b677f44 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:21.103445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 222/446] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:21.104259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4436e-56ff-45a1-aff3-9aff4b677f44 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:21.109279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc4a2c-c57f-4afe-b81b-5853b8391ba2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:21.144590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad2ed5-6682-414e-8a81-aee37b0d0147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:21.145216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2ed5-6682-414e-8a81-aee37b0d0147 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:21.145498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2ed5-6682-414e-8a81-aee37b0d0147 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:21.145772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2ed5-6682-414e-8a81-aee37b0d0147 None admin] PATH_INFO: `/v3/projects/10e7858e00034a8284453b08e8470748` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:21.146929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad2ed5-6682-414e-8a81-aee37b0d0147 None admin] RBAC: Authorizing `identity:get_project(project_id=10e7858e00034a8284453b08e8470748)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:21.148413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad2ed5-6682-414e-8a81-aee37b0d0147 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:21.150042 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 225/447] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:21 2022] GET /identity/v3/projects/10e7858e00034a8284453b08e8470748 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:21.150798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad2ed5-6682-414e-8a81-aee37b0d0147 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:21.153127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:21.161971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:21.209712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4436e-56ff-45a1-aff3-9aff4b677f44 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:21.243438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcadda9-206f-4f3d-9166-21d017e45a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:21.244038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcadda9-206f-4f3d-9166-21d017e45a14 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:21.244346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcadda9-206f-4f3d-9166-21d017e45a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:21.244620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcadda9-206f-4f3d-9166-21d017e45a14 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:21.245275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcadda9-206f-4f3d-9166-21d017e45a14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:21.245578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcadda9-206f-4f3d-9166-21d017e45a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:21.246941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcadda9-206f-4f3d-9166-21d017e45a14 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:21.252950 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad2ed5-6682-414e-8a81-aee37b0d0147 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:21.255223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 223/448] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:21.256020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcadda9-206f-4f3d-9166-21d017e45a14 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:21.286880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843976e-9957-45ee-a17d-087939331e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:21.287502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843976e-9957-45ee-a17d-087939331e0e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:21.287777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843976e-9957-45ee-a17d-087939331e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:21.288080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843976e-9957-45ee-a17d-087939331e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:21.288789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843976e-9957-45ee-a17d-087939331e0e None admin] RBAC: Adding query filter params (group.id=29e80051419f49bc97ff5afbea57843b, role.id=7ab2687826544b8e80c709d6061162d4, scope.project.id=10e7858e00034a8284453b08e8470748) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:21.289096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843976e-9957-45ee-a17d-087939331e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:21.290525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843976e-9957-45ee-a17d-087939331e0e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:21.296349 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 226/449] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Aug 24 09:50:21 2022] GET /identity/v3/role_assignments?group.id=29e80051419f49bc97ff5afbea57843b&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=7ab2687826544b8e80c709d6061162d4 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:21.297111 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843976e-9957-45ee-a17d-087939331e0e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:21.366441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcadda9-206f-4f3d-9166-21d017e45a14 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:21.374126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:21.384897 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:21.407562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb055e-d872-47e0-8438-a2d162cf74e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:21.408251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb055e-d872-47e0-8438-a2d162cf74e6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:21.408558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb055e-d872-47e0-8438-a2d162cf74e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:21.408807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb055e-d872-47e0-8438-a2d162cf74e6 None admin] PATH_INFO: `/v3/groups/29e80051419f49bc97ff5afbea57843b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:21.410266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb055e-d872-47e0-8438-a2d162cf74e6 None admin] RBAC: Authorizing `identity:get_group(group_id=29e80051419f49bc97ff5afbea57843b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:21.411743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb055e-d872-47e0-8438-a2d162cf74e6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:21.414092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 224/450] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:21 2022] GET /identity/v3/groups/29e80051419f49bc97ff5afbea57843b => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:21.419509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843976e-9957-45ee-a17d-087939331e0e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:21.419876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb055e-d872-47e0-8438-a2d162cf74e6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:21.458538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06963dd3-1e09-4ed8-b43c-c83f081a4f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:21.459178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06963dd3-1e09-4ed8-b43c-c83f081a4f2c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:21.459488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06963dd3-1e09-4ed8-b43c-c83f081a4f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:21.459818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06963dd3-1e09-4ed8-b43c-c83f081a4f2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:21.541558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb055e-d872-47e0-8438-a2d162cf74e6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:21.544235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06963dd3-1e09-4ed8-b43c-c83f081a4f2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:21.545956 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06963dd3-1e09-4ed8-b43c-c83f081a4f2c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:21.581807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e5fe6-e8a1-4c74-b7f6-32cd6789b740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:21.582235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e5fe6-e8a1-4c74-b7f6-32cd6789b740 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:21.582520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e5fe6-e8a1-4c74-b7f6-32cd6789b740 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:21.582801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e5fe6-e8a1-4c74-b7f6-32cd6789b740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:21.583542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e5fe6-e8a1-4c74-b7f6-32cd6789b740 None admin] RBAC: Adding query filter params (group.id=29e80051419f49bc97ff5afbea57843b, role.id=fc1186ecd0f74b2fad8155d528f268f9, scope.project.id=10e7858e00034a8284453b08e8470748) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:21.583852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e5fe6-e8a1-4c74-b7f6-32cd6789b740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:21.586405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e5fe6-e8a1-4c74-b7f6-32cd6789b740 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:21.593016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 225/451] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Aug 24 09:50:21 2022] GET /identity/v3/role_assignments?group.id=29e80051419f49bc97ff5afbea57843b&scope.project.id=10e7858e00034a8284453b08e8470748&role.id=fc1186ecd0f74b2fad8155d528f268f9 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:21.635664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:21.671708 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 227/452] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:21.677727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e5fe6-e8a1-4c74-b7f6-32cd6789b740 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:21.720443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:21.788863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e5fe6-e8a1-4c74-b7f6-32cd6789b740 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:21.823253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cc270-93e2-49c0-ba32-f9066d28fb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:21.823882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc270-93e2-49c0-ba32-f9066d28fb0f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:21.824197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc270-93e2-49c0-ba32-f9066d28fb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:21.824474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cc270-93e2-49c0-ba32-f9066d28fb0f None admin] PATH_INFO: `/v3/projects/f71a167399734714a60b777eea4e2ecc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:21.825604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cc270-93e2-49c0-ba32-f9066d28fb0f None admin] RBAC: Authorizing `identity:get_project(project_id=f71a167399734714a60b777eea4e2ecc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:21.827056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cc270-93e2-49c0-ba32-f9066d28fb0f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:21.828728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 226/453] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:50:21 2022] GET /identity/v3/projects/f71a167399734714a60b777eea4e2ecc => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:21.835009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06963dd3-1e09-4ed8-b43c-c83f081a4f2c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:21.938951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06963dd3-1e09-4ed8-b43c-c83f081a4f2c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:21.972167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4df5b-aba6-4867-8171-db85606036bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:21.972703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4df5b-aba6-4867-8171-db85606036bb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:21.972981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4df5b-aba6-4867-8171-db85606036bb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:21.973252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4df5b-aba6-4867-8171-db85606036bb None admin] PATH_INFO: `/v3/groups/68e6cd5887ec4e018b0e8d72ef6d48b1` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:21.974343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4df5b-aba6-4867-8171-db85606036bb None admin] RBAC: Authorizing `identity:get_group(group_id=68e6cd5887ec4e018b0e8d72ef6d48b1)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:21.976426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4df5b-aba6-4867-8171-db85606036bb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:21.977980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 228/454] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Aug 24 09:50:21 2022] GET /identity/v3/groups/68e6cd5887ec4e018b0e8d72ef6d48b1 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:21.983828 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cc270-93e2-49c0-ba32-f9066d28fb0f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:22.080644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cc270-93e2-49c0-ba32-f9066d28fb0f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:22.087718 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:22.114320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494b668-5377-4e7d-bb29-5fd196b06bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:22.114908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494b668-5377-4e7d-bb29-5fd196b06bcb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:22.115190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494b668-5377-4e7d-bb29-5fd196b06bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:22.115465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494b668-5377-4e7d-bb29-5fd196b06bcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:22.116241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494b668-5377-4e7d-bb29-5fd196b06bcb None admin] RBAC: Adding query filter params (group.id=68e6cd5887ec4e018b0e8d72ef6d48b1, role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=f71a167399734714a60b777eea4e2ecc) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:22.116551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494b668-5377-4e7d-bb29-5fd196b06bcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:22.117995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494b668-5377-4e7d-bb29-5fd196b06bcb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:22.123871 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 227/455] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Aug 24 09:50:21 2022] GET /identity/v3/role_assignments?group.id=68e6cd5887ec4e018b0e8d72ef6d48b1&scope.project.id=f71a167399734714a60b777eea4e2ecc&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:24.513718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d662fc-dfb6-4a34-9e08-38d93d32d0b5 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:24.514233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d662fc-dfb6-4a34-9e08-38d93d32d0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:24.514233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d662fc-dfb6-4a34-9e08-38d93d32d0b5 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:24.515879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 229/456] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:24.521097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163ec6-aadf-429c-8f2b-ea92cf106bbe None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:24.521499 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163ec6-aadf-429c-8f2b-ea92cf106bbe None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:24.521665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163ec6-aadf-429c-8f2b-ea92cf106bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:24.549066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e8163ec6-aadf-429c-8f2b-ea92cf106bbe None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:24.550278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e8163ec6-aadf-429c-8f2b-ea92cf106bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:24.568696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f18ad-abc0-467d-8f93-a041d5e9725d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:24.569044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f18ad-abc0-467d-8f93-a041d5e9725d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:24.569290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f18ad-abc0-467d-8f93-a041d5e9725d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:24.570917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 230/457] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:24.576460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5100bef-053f-4883-9141-1508232bab35 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:24.576739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5100bef-053f-4883-9141-1508232bab35 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:24.577031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5100bef-053f-4883-9141-1508232bab35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:24.603711 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d5100bef-053f-4883-9141-1508232bab35 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:24.604935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d5100bef-053f-4883-9141-1508232bab35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:24.667138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 228/458] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:24.692412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:24.722638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 231/459] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:24.745054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fb60a-a328-4573-b1da-595a63b97420 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:24.745353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fb60a-a328-4573-b1da-595a63b97420 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:24.745606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fb60a-a328-4573-b1da-595a63b97420 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:24.746962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 229/460] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:24.752190 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348722f1-70f0-4bd9-a4a6-111499d89c15 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:24.752478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348722f1-70f0-4bd9-a4a6-111499d89c15 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:24.752753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348722f1-70f0-4bd9-a4a6-111499d89c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:24.781069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-348722f1-70f0-4bd9-a4a6-111499d89c15 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:24.782266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-348722f1-70f0-4bd9-a4a6-111499d89c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:24.865470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 232/461] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:24.983212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fb60a-a328-4573-b1da-595a63b97420 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:25.080014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348722f1-70f0-4bd9-a4a6-111499d89c15 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:25.120842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:25.131681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fb60a-a328-4573-b1da-595a63b97420 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:25.200937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4da8d-5ce9-496c-b6c6-333b998f27da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:25.201566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4da8d-5ce9-496c-b6c6-333b998f27da None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.201843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4da8d-5ce9-496c-b6c6-333b998f27da None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.202112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4da8d-5ce9-496c-b6c6-333b998f27da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.221132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4da8d-5ce9-496c-b6c6-333b998f27da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:25.222942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4da8d-5ce9-496c-b6c6-333b998f27da None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:25.248652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:25.252277 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348722f1-70f0-4bd9-a4a6-111499d89c15 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-26e4da8d-5ce9-496c-b6c6-333b998f27da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:25.253781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:25.254722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:25.255563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:25.255563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:25.255563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:25.255563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 230/462] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:25.257493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4da8d-5ce9-496c-b6c6-333b998f27da None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:25.300492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db2a6a-f780-4028-8265-cee7d733fa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:25.301771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db2a6a-f780-4028-8265-cee7d733fa9c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.302297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db2a6a-f780-4028-8265-cee7d733fa9c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.303588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db2a6a-f780-4028-8265-cee7d733fa9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.322821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db2a6a-f780-4028-8265-cee7d733fa9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:25.325141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db2a6a-f780-4028-8265-cee7d733fa9c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-03db2a6a-f780-4028-8265-cee7d733fa9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:25.347143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:25.348097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:25.348987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:25.348987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:25.348987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:25.350651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 233/463] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:25.352362 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db2a6a-f780-4028-8265-cee7d733fa9c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:25.390987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4da8d-5ce9-496c-b6c6-333b998f27da None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:25.441194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993735f5-db4a-44e9-a315-8dd99f984572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:25.443164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993735f5-db4a-44e9-a315-8dd99f984572 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.443626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993735f5-db4a-44e9-a315-8dd99f984572 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.444084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993735f5-db4a-44e9-a315-8dd99f984572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.460498 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993735f5-db4a-44e9-a315-8dd99f984572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:25.462082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993735f5-db4a-44e9-a315-8dd99f984572 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-993735f5-db4a-44e9-a315-8dd99f984572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:25.477266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:25.478227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:25.479053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:25.479053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:25.479053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:25.483287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 231/464] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:25.484225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993735f5-db4a-44e9-a315-8dd99f984572 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:25.484344 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db2a6a-f780-4028-8265-cee7d733fa9c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:25.487969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:25.526752 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2ad09-4423-439f-af43-3ba8ddc8ace2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:25.527435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2ad09-4423-439f-af43-3ba8ddc8ace2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.527635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2ad09-4423-439f-af43-3ba8ddc8ace2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.527905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2ad09-4423-439f-af43-3ba8ddc8ace2 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.528941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2ad09-4423-439f-af43-3ba8ddc8ace2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:25.529250 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2ad09-4423-439f-af43-3ba8ddc8ace2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:25.531264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2ad09-4423-439f-af43-3ba8ddc8ace2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:25.544978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 234/465] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:25.546590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8761457-1c0c-4c97-9616-3d2ca6083559 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.546703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8761457-1c0c-4c97-9616-3d2ca6083559 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.546953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8761457-1c0c-4c97-9616-3d2ca6083559 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.552175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 235/466] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:25.552175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8761457-1c0c-4c97-9616-3d2ca6083559 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:25.567261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:25.603604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993735f5-db4a-44e9-a315-8dd99f984572 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:25.641362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55668483-6b3d-419b-8fa5-6a149d3ce2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:25.642210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55668483-6b3d-419b-8fa5-6a149d3ce2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.642720 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55668483-6b3d-419b-8fa5-6a149d3ce2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.643140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55668483-6b3d-419b-8fa5-6a149d3ce2b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.643930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55668483-6b3d-419b-8fa5-6a149d3ce2b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:25.644402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55668483-6b3d-419b-8fa5-6a149d3ce2b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:25.645929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55668483-6b3d-419b-8fa5-6a149d3ce2b9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:25.657639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8761457-1c0c-4c97-9616-3d2ca6083559 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:25.659144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 232/467] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:25.660121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55668483-6b3d-419b-8fa5-6a149d3ce2b9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:25.698475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47308839-7e42-430a-b6cb-ea1c87b92540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:25.699074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47308839-7e42-430a-b6cb-ea1c87b92540 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.699355 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47308839-7e42-430a-b6cb-ea1c87b92540 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.699634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47308839-7e42-430a-b6cb-ea1c87b92540 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.700297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47308839-7e42-430a-b6cb-ea1c87b92540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:25.700605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47308839-7e42-430a-b6cb-ea1c87b92540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:25.701990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47308839-7e42-430a-b6cb-ea1c87b92540 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:25.718077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 236/468] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:25.718077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51808555-0360-4103-a668-81016f65c41d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.718077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51808555-0360-4103-a668-81016f65c41d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.718077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51808555-0360-4103-a668-81016f65c41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.733589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-51808555-0360-4103-a668-81016f65c41d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:25.734444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:25.734925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-51808555-0360-4103-a668-81016f65c41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:25.779521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55668483-6b3d-419b-8fa5-6a149d3ce2b9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:25.817922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 237/469] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:25.817922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51808555-0360-4103-a668-81016f65c41d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:25.817922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:25.824067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d0174-aac1-47ac-a1e4-b28c694e29c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:25.824067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d0174-aac1-47ac-a1e4-b28c694e29c5 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.824067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d0174-aac1-47ac-a1e4-b28c694e29c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.824067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d0174-aac1-47ac-a1e4-b28c694e29c5 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.824413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d0174-aac1-47ac-a1e4-b28c694e29c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:25.825350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d0174-aac1-47ac-a1e4-b28c694e29c5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:25.850890 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 233/470] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:50:25 2022] POST /identity/v3/users => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:25.851777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d0174-aac1-47ac-a1e4-b28c694e29c5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:25.966609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51808555-0360-4103-a668-81016f65c41d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:25.996310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d0174-aac1-47ac-a1e4-b28c694e29c5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:26.006283 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a04c5-1d33-4f7a-b6b6-4bdeaea9f497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:26.006954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a04c5-1d33-4f7a-b6b6-4bdeaea9f497 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:26.007235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a04c5-1d33-4f7a-b6b6-4bdeaea9f497 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:26.007509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a04c5-1d33-4f7a-b6b6-4bdeaea9f497 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:26.008206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a04c5-1d33-4f7a-b6b6-4bdeaea9f497 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:26.009813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a04c5-1d33-4f7a-b6b6-4bdeaea9f497 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:26.021433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:26.040159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 238/471] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:50:25 2022] POST /identity/v3/users => generated 248 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:26.041658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dfc88-0f60-41cb-9b32-c7992299c534 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:26.041947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dfc88-0f60-41cb-9b32-c7992299c534 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:26.042225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dfc88-0f60-41cb-9b32-c7992299c534 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:26.048243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 239/472] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:26.051457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2185551-3e0c-48da-a30c-fd0377109c81 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:26.051739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2185551-3e0c-48da-a30c-fd0377109c81 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:26.052066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2185551-3e0c-48da-a30c-fd0377109c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:26.056478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957c390-88e4-4e80-aa7b-3e9a2da306a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:26.057569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957c390-88e4-4e80-aa7b-3e9a2da306a8 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:26.058091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957c390-88e4-4e80-aa7b-3e9a2da306a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:26.058967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957c390-88e4-4e80-aa7b-3e9a2da306a8 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:26.064627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957c390-88e4-4e80-aa7b-3e9a2da306a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:26.066162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957c390-88e4-4e80-aa7b-3e9a2da306a8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:26.077136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f2185551-3e0c-48da-a30c-fd0377109c81 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:26.080088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f2185551-3e0c-48da-a30c-fd0377109c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:26.096302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 234/473] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:50:25 2022] POST /identity/v3/users => generated 248 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:26.148774 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957c390-88e4-4e80-aa7b-3e9a2da306a8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:26.159598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:26.181753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 240/474] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:26.288665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957c390-88e4-4e80-aa7b-3e9a2da306a8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:26.325595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3339a4-86f0-4b0f-9742-5ba637cc7a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:26.326225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3339a4-86f0-4b0f-9742-5ba637cc7a0c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:26.326503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3339a4-86f0-4b0f-9742-5ba637cc7a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:26.326776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3339a4-86f0-4b0f-9742-5ba637cc7a0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:26.338500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3339a4-86f0-4b0f-9742-5ba637cc7a0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:26.339970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3339a4-86f0-4b0f-9742-5ba637cc7a0c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ce3339a4-86f0-4b0f-9742-5ba637cc7a0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:26.355114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:26.356128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:26.357027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:26.357027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:26.357027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:26.357027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 235/475] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:26.363355 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2185551-3e0c-48da-a30c-fd0377109c81 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:26.469914 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2185551-3e0c-48da-a30c-fd0377109c81 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:26.486622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3339a4-86f0-4b0f-9742-5ba637cc7a0c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:26.506289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b517d-06d1-4a9a-a5c1-ec0f93ea953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:26.507800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b517d-06d1-4a9a-a5c1-ec0f93ea953d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:26.508312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b517d-06d1-4a9a-a5c1-ec0f93ea953d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:26.508797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b517d-06d1-4a9a-a5c1-ec0f93ea953d None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:26.509649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b517d-06d1-4a9a-a5c1-ec0f93ea953d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:26.510148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b517d-06d1-4a9a-a5c1-ec0f93ea953d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:26.511748 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b517d-06d1-4a9a-a5c1-ec0f93ea953d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:26.519147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:26.528765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 241/476] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:26.530490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b517d-06d1-4a9a-a5c1-ec0f93ea953d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:26.588873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:26.636880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b517d-06d1-4a9a-a5c1-ec0f93ea953d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:26.650506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3339a4-86f0-4b0f-9742-5ba637cc7a0c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:26.674475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff257994-5325-4df2-9b22-280037c42621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:26.675082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff257994-5325-4df2-9b22-280037c42621 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:26.675362 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff257994-5325-4df2-9b22-280037c42621 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:26.675634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff257994-5325-4df2-9b22-280037c42621 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:26.676309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff257994-5325-4df2-9b22-280037c42621 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:26.677750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff257994-5325-4df2-9b22-280037c42621 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:26.707892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 242/477] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:50:26 2022] POST /identity/v3/users => generated 249 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:26.749575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2fb7a-9ec4-472e-bc39-12ee602e928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:26.750210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2fb7a-9ec4-472e-bc39-12ee602e928c None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:26.750494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2fb7a-9ec4-472e-bc39-12ee602e928c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:26.750775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2fb7a-9ec4-472e-bc39-12ee602e928c None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:26.751492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2fb7a-9ec4-472e-bc39-12ee602e928c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:26.752981 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2fb7a-9ec4-472e-bc39-12ee602e928c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:26.768601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.sql.core [None req-e4d2fb7a-9ec4-472e-bc39-12ee602e928c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:50:26.768601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 09:50:26.768601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [parameters: {'id': '7eec9ed393714f019a2a11ecd7b86234', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 09:50:26.768601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81463) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e4d2fb7a-9ec4-472e-bc39-12ee602e928c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Aug 24 09:50:26.792703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:50:26.793649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application yield resource Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise self.value Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application session.commit() Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:50:26.794571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:50:26.795476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:50:26.796660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:50:26.797489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [parameters: {'id': '7eec9ed393714f019a2a11ecd7b86234', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:26.798297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 09:50:26.799106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:26.806258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 236/478] 10.222.0.86 () {60 vars in 1131 bytes} [Wed Aug 24 09:50:26 2022] POST /identity/v3/roles => generated 144 bytes in 308 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:26.809546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff257994-5325-4df2-9b22-280037c42621 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:26.929412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff257994-5325-4df2-9b22-280037c42621 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:26.931681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:26.965292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a505f2-3dcb-425d-ad4a-1be1259273a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:26.966000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a505f2-3dcb-425d-ad4a-1be1259273a6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:26.966284 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a505f2-3dcb-425d-ad4a-1be1259273a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:26.966555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a505f2-3dcb-425d-ad4a-1be1259273a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:26.978399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a505f2-3dcb-425d-ad4a-1be1259273a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-59a505f2-3dcb-425d-ad4a-1be1259273a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:26.983688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:26.984587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:26.985388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 243/479] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:50:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:26.992820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2fb7a-9ec4-472e-bc39-12ee602e928c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:27.095162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:27.104983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2fb7a-9ec4-472e-bc39-12ee602e928c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:27.140363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d60c7d-82f4-4fd1-bba4-8b097025cc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:27.140975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d60c7d-82f4-4fd1-bba4-8b097025cc8d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:27.141258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d60c7d-82f4-4fd1-bba4-8b097025cc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:27.141534 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d60c7d-82f4-4fd1-bba4-8b097025cc8d None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:27.142204 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d60c7d-82f4-4fd1-bba4-8b097025cc8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:27.142512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d60c7d-82f4-4fd1-bba4-8b097025cc8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:27.143880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d60c7d-82f4-4fd1-bba4-8b097025cc8d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:27.152224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 237/480] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:28.361781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919a62d-d475-4b18-81bd-0b9a7dcca53a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:28.362301 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919a62d-d475-4b18-81bd-0b9a7dcca53a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:28.362378 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919a62d-d475-4b18-81bd-0b9a7dcca53a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:28.363939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 244/481] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:28.371262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c7f43-b5c1-47b9-ab8c-cfb944df6bde None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:28.371693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c7f43-b5c1-47b9-ab8c-cfb944df6bde None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:28.371866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c7f43-b5c1-47b9-ab8c-cfb944df6bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:28.393208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-715c7f43-b5c1-47b9-ab8c-cfb944df6bde None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:28.394415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-715c7f43-b5c1-47b9-ab8c-cfb944df6bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:28.485656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 238/482] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:28.500172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74485987-876b-4b87-893a-3ee0837b8911 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:28.500520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74485987-876b-4b87-893a-3ee0837b8911 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:28.500725 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74485987-876b-4b87-893a-3ee0837b8911 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:28.502292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 245/483] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:28.513461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3578f-ca93-4847-877b-c1672311cc51 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:28.513748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3578f-ca93-4847-877b-c1672311cc51 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:28.514027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3578f-ca93-4847-877b-c1672311cc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:28.535791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-79d3578f-ca93-4847-877b-c1672311cc51 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:28.537014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-79d3578f-ca93-4847-877b-c1672311cc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:28.579183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10083f39-715f-46c8-a007-8a5f005a294c None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:28.579806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10083f39-715f-46c8-a007-8a5f005a294c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:28.580333 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10083f39-715f-46c8-a007-8a5f005a294c None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:28.582128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 246/484] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:28.587973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336c225-16e1-4a0e-9d61-77526c5d0f99 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:28.588514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336c225-16e1-4a0e-9d61-77526c5d0f99 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:28.589004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336c225-16e1-4a0e-9d61-77526c5d0f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:28.617606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d336c225-16e1-4a0e-9d61-77526c5d0f99 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:28.618730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d336c225-16e1-4a0e-9d61-77526c5d0f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:28.651201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 239/485] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:28.706973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 247/486] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:28.859942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3578f-ca93-4847-877b-c1672311cc51 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:28.879172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:28.993962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3578f-ca93-4847-877b-c1672311cc51 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.036627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef58e29-028c-4bf0-ac8b-7b50a8d17133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.037255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef58e29-028c-4bf0-ac8b-7b50a8d17133 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.037539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef58e29-028c-4bf0-ac8b-7b50a8d17133 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.037820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef58e29-028c-4bf0-ac8b-7b50a8d17133 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.056916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef58e29-028c-4bf0-ac8b-7b50a8d17133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-cef58e29-028c-4bf0-ac8b-7b50a8d17133 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:29.061829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:29.062754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:29.063965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 240/487] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:29.069068 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336c225-16e1-4a0e-9d61-77526c5d0f99 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.089150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef58e29-028c-4bf0-ac8b-7b50a8d17133 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.184174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:29.204489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:29.208585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336c225-16e1-4a0e-9d61-77526c5d0f99 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.211822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef58e29-028c-4bf0-ac8b-7b50a8d17133 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.248980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712754b-5fdd-4ffe-8f2f-99e5655d1ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.249750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712754b-5fdd-4ffe-8f2f-99e5655d1ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.250126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712754b-5fdd-4ffe-8f2f-99e5655d1ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.250485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712754b-5fdd-4ffe-8f2f-99e5655d1ba6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.253201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd89fb-052b-4f23-a202-7752f02c339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.253977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd89fb-052b-4f23-a202-7752f02c339a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.254399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd89fb-052b-4f23-a202-7752f02c339a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.254870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd89fb-052b-4f23-a202-7752f02c339a None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.255677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd89fb-052b-4f23-a202-7752f02c339a None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:29.256166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd89fb-052b-4f23-a202-7752f02c339a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:29.258226 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd89fb-052b-4f23-a202-7752f02c339a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:29.262247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712754b-5fdd-4ffe-8f2f-99e5655d1ba6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7712754b-5fdd-4ffe-8f2f-99e5655d1ba6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:29.267205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:29.268178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:29.268983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 241/488] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:29.269657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712754b-5fdd-4ffe-8f2f-99e5655d1ba6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.277462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 248/489] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:29.278559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd89fb-052b-4f23-a202-7752f02c339a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.379635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712754b-5fdd-4ffe-8f2f-99e5655d1ba6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.384746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd89fb-052b-4f23-a202-7752f02c339a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.418713 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33ba75-ab69-4759-9b69-9c2f28cbb49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.419342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33ba75-ab69-4759-9b69-9c2f28cbb49d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.419638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33ba75-ab69-4759-9b69-9c2f28cbb49d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.419899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33ba75-ab69-4759-9b69-9c2f28cbb49d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.422473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:29.424868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4174ac0-7883-4025-b1f0-c155e3761d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.425651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4174ac0-7883-4025-b1f0-c155e3761d6d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.425919 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4174ac0-7883-4025-b1f0-c155e3761d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.426264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4174ac0-7883-4025-b1f0-c155e3761d6d None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.426952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4174ac0-7883-4025-b1f0-c155e3761d6d None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:29.427258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4174ac0-7883-4025-b1f0-c155e3761d6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:29.428693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4174ac0-7883-4025-b1f0-c155e3761d6d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:29.433859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33ba75-ab69-4759-9b69-9c2f28cbb49d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:29.438181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 249/490] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:29.438839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4174ac0-7883-4025-b1f0-c155e3761d6d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-df33ba75-ab69-4759-9b69-9c2f28cbb49d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:29.442876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:29.443755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:29.444607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 242/491] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:29.445649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a3cb6-ffc4-48b6-bb1f-ee3a045079a5 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.445649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a3cb6-ffc4-48b6-bb1f-ee3a045079a5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.445861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a3cb6-ffc4-48b6-bb1f-ee3a045079a5 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.447277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 243/492] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:29.447993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a3cb6-ffc4-48b6-bb1f-ee3a045079a5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.464637 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:29.556802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4174ac0-7883-4025-b1f0-c155e3761d6d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.566284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a3cb6-ffc4-48b6-bb1f-ee3a045079a5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.593138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8d35f-45bc-4480-a3a7-bc1f546b9343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.593675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d35f-45bc-4480-a3a7-bc1f546b9343 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.593953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d35f-45bc-4480-a3a7-bc1f546b9343 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.594223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d35f-45bc-4480-a3a7-bc1f546b9343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.602739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed0e43-0ac2-40e3-8788-bf00e50de46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.603360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed0e43-0ac2-40e3-8788-bf00e50de46c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.603963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed0e43-0ac2-40e3-8788-bf00e50de46c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.603963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed0e43-0ac2-40e3-8788-bf00e50de46c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.605980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8d35f-45bc-4480-a3a7-bc1f546b9343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:29.607430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8d35f-45bc-4480-a3a7-bc1f546b9343 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:29.614743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed0e43-0ac2-40e3-8788-bf00e50de46c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:29.616214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed0e43-0ac2-40e3-8788-bf00e50de46c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-21b8d35f-45bc-4480-a3a7-bc1f546b9343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:29.622351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:29.623228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:29.624938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:29.624938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:29.624938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:29.624938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 250/493] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:29.625480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29364d-30c1-42f3-826c-bbc065181f6a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.625776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29364d-30c1-42f3-826c-bbc065181f6a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.626049 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29364d-30c1-42f3-826c-bbc065181f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-17ed0e43-0ac2-40e3-8788-bf00e50de46c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:29.636108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:29.637051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:29.637898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:29.637898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:29.637898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:29.637898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 244/494] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:29.638547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed0e43-0ac2-40e3-8788-bf00e50de46c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.645424 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-eb29364d-30c1-42f3-826c-bbc065181f6a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:29.646598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-eb29364d-30c1-42f3-826c-bbc065181f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:29.693062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:29.718941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:29.736075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 251/495] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:29.736075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29364d-30c1-42f3-826c-bbc065181f6a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.768189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed0e43-0ac2-40e3-8788-bf00e50de46c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.805818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54238441-f650-4a01-abe2-a368d8934bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.806630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54238441-f650-4a01-abe2-a368d8934bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.807096 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54238441-f650-4a01-abe2-a368d8934bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.807471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54238441-f650-4a01-abe2-a368d8934bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.808316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54238441-f650-4a01-abe2-a368d8934bf6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:29.808721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54238441-f650-4a01-abe2-a368d8934bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:29.811167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54238441-f650-4a01-abe2-a368d8934bf6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:29.821843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 245/496] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:29.822804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54238441-f650-4a01-abe2-a368d8934bf6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.853981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29364d-30c1-42f3-826c-bbc065181f6a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.891051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df87a0-8fb2-4b1f-b121-1ba930979a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.891778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df87a0-8fb2-4b1f-b121-1ba930979a02 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.892075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df87a0-8fb2-4b1f-b121-1ba930979a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.892372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df87a0-8fb2-4b1f-b121-1ba930979a02 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.893038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df87a0-8fb2-4b1f-b121-1ba930979a02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:29.893327 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df87a0-8fb2-4b1f-b121-1ba930979a02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:29.894837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df87a0-8fb2-4b1f-b121-1ba930979a02 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:29.911818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 252/497] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:29.911818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5159703-d4c6-4031-9179-c581472a0b8c None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.911818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5159703-d4c6-4031-9179-c581472a0b8c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.911818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5159703-d4c6-4031-9179-c581472a0b8c None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.911818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 253/498] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:29.912797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5159703-d4c6-4031-9179-c581472a0b8c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.932865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54238441-f650-4a01-abe2-a368d8934bf6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.941977 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:29.970427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603e7d-0a13-4d2b-ac73-43a705cff21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.970945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603e7d-0a13-4d2b-ac73-43a705cff21b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.971222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603e7d-0a13-4d2b-ac73-43a705cff21b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.971493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603e7d-0a13-4d2b-ac73-43a705cff21b None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.972171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76603e7d-0a13-4d2b-ac73-43a705cff21b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:29.972478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76603e7d-0a13-4d2b-ac73-43a705cff21b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:29.973852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76603e7d-0a13-4d2b-ac73-43a705cff21b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:29.986125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 246/499] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:29.986922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603e7d-0a13-4d2b-ac73-43a705cff21b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.992944 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:30.037291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5159703-d4c6-4031-9179-c581472a0b8c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.072776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cf145-e084-4ac7-85b4-d6f73d1b3c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.073395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf145-e084-4ac7-85b4-d6f73d1b3c91 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.073678 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf145-e084-4ac7-85b4-d6f73d1b3c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.073953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf145-e084-4ac7-85b4-d6f73d1b3c91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.084246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cf145-e084-4ac7-85b4-d6f73d1b3c91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:30.086229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cf145-e084-4ac7-85b4-d6f73d1b3c91 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:30.087007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603e7d-0a13-4d2b-ac73-43a705cff21b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-603cf145-e084-4ac7-85b4-d6f73d1b3c91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:30.099930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:30.100962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:30.102782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:30.102782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:30.102782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:30.102782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 254/500] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:30.102782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad61a8-02d5-4940-8d31-af9f183c11b1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.103069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad61a8-02d5-4940-8d31-af9f183c11b1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.103275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad61a8-02d5-4940-8d31-af9f183c11b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.120512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-59ad61a8-02d5-4940-8d31-af9f183c11b1 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:30.121750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-59ad61a8-02d5-4940-8d31-af9f183c11b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:30.127050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cb93d-ef4c-42b0-a9e3-eb8ade4a48da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.127667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cb93d-ef4c-42b0-a9e3-eb8ade4a48da None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.127947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cb93d-ef4c-42b0-a9e3-eb8ade4a48da None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.128250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cb93d-ef4c-42b0-a9e3-eb8ade4a48da None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.135504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cb93d-ef4c-42b0-a9e3-eb8ade4a48da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:30.136746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cb93d-ef4c-42b0-a9e3-eb8ade4a48da None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a00cb93d-ef4c-42b0-a9e3-eb8ade4a48da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:30.149739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:30.150629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 247/501] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:30.151561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cb93d-ef4c-42b0-a9e3-eb8ade4a48da None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.183273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:30.204070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 255/502] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:30.204070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad61a8-02d5-4940-8d31-af9f183c11b1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.237979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:30.258704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cb93d-ef4c-42b0-a9e3-eb8ade4a48da None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.293958 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb1e5b-ec1e-49f5-86c8-bc4496255c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.294577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1e5b-ec1e-49f5-86c8-bc4496255c90 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.294859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1e5b-ec1e-49f5-86c8-bc4496255c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.295133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1e5b-ec1e-49f5-86c8-bc4496255c90 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.299152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad61a8-02d5-4940-8d31-af9f183c11b1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.303231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb1e5b-ec1e-49f5-86c8-bc4496255c90 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:30.305337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb1e5b-ec1e-49f5-86c8-bc4496255c90 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e2fb1e5b-ec1e-49f5-86c8-bc4496255c90 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:30.318321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:30.319246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:30.320911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:30.320911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:30.320911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:30.320911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:30.320911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:30.320911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:30.320911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:30.320911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:30.320911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:50:30.320911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:30.324874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 248/503] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:30.325635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb1e5b-ec1e-49f5-86c8-bc4496255c90 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.338160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc773668-804b-4cb1-a7aa-cd0f13defbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.338917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc773668-804b-4cb1-a7aa-cd0f13defbec None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.339108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc773668-804b-4cb1-a7aa-cd0f13defbec None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.339410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc773668-804b-4cb1-a7aa-cd0f13defbec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.350059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc773668-804b-4cb1-a7aa-cd0f13defbec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-cc773668-804b-4cb1-a7aa-cd0f13defbec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:30.354774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:30.355679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:30.356546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 256/504] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:30.357675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc773668-804b-4cb1-a7aa-cd0f13defbec None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.431944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:30.448246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb1e5b-ec1e-49f5-86c8-bc4496255c90 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.468095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc773668-804b-4cb1-a7aa-cd0f13defbec None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.469818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:30.485419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4729233-893b-4066-83bf-42956bc529f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.486019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4729233-893b-4066-83bf-42956bc529f1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.486295 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4729233-893b-4066-83bf-42956bc529f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.486564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4729233-893b-4066-83bf-42956bc529f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.487200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4729233-893b-4066-83bf-42956bc529f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:30.487500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4729233-893b-4066-83bf-42956bc529f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:30.488935 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4729233-893b-4066-83bf-42956bc529f1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:30.501163 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 249/505] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:30.501966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4729233-893b-4066-83bf-42956bc529f1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.505438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68b147-c32a-43d4-929e-3414b5524480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.506058 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68b147-c32a-43d4-929e-3414b5524480 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.506341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68b147-c32a-43d4-929e-3414b5524480 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.506612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68b147-c32a-43d4-929e-3414b5524480 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.507333 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68b147-c32a-43d4-929e-3414b5524480 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:30.507679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68b147-c32a-43d4-929e-3414b5524480 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:30.509158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68b147-c32a-43d4-929e-3414b5524480 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:30.544892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 257/506] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Aug 24 09:50:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:30.545711 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68b147-c32a-43d4-929e-3414b5524480 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.604385 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4729233-893b-4066-83bf-42956bc529f1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.639539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e0dae-a85f-4177-86d6-11f45af360f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.640165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e0dae-a85f-4177-86d6-11f45af360f9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.640534 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e0dae-a85f-4177-86d6-11f45af360f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.640806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e0dae-a85f-4177-86d6-11f45af360f9 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.641535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e0dae-a85f-4177-86d6-11f45af360f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:30.641844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e0dae-a85f-4177-86d6-11f45af360f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:30.643275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e0dae-a85f-4177-86d6-11f45af360f9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:30.644324 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68b147-c32a-43d4-929e-3414b5524480 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.680800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 250/507] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Aug 24 09:50:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:30.681634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e0dae-a85f-4177-86d6-11f45af360f9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.686161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a46aab-6aab-4652-b0d6-6d82a2059a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.686765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a46aab-6aab-4652-b0d6-6d82a2059a8b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.687047 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a46aab-6aab-4652-b0d6-6d82a2059a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.687321 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a46aab-6aab-4652-b0d6-6d82a2059a8b None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.688451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a46aab-6aab-4652-b0d6-6d82a2059a8b None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:30.689017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a46aab-6aab-4652-b0d6-6d82a2059a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:30.690472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a46aab-6aab-4652-b0d6-6d82a2059a8b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:30.693528 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:30.698931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 258/508] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:30.699733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a46aab-6aab-4652-b0d6-6d82a2059a8b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.723059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:30.792152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e0dae-a85f-4177-86d6-11f45af360f9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.801740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a46aab-6aab-4652-b0d6-6d82a2059a8b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.825774 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb0840-d545-4452-a244-bc4da75ee45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.826388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb0840-d545-4452-a244-bc4da75ee45c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.826665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb0840-d545-4452-a244-bc4da75ee45c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.826936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb0840-d545-4452-a244-bc4da75ee45c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.834231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df905377-2cc7-48cb-8470-50340992e256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.834855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df905377-2cc7-48cb-8470-50340992e256 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.835131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df905377-2cc7-48cb-8470-50340992e256 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.835402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df905377-2cc7-48cb-8470-50340992e256 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.837318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb0840-d545-4452-a244-bc4da75ee45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:30.838770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb0840-d545-4452-a244-bc4da75ee45c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:30.843256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df905377-2cc7-48cb-8470-50340992e256 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:30.844735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df905377-2cc7-48cb-8470-50340992e256 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8dcb0840-d545-4452-a244-bc4da75ee45c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:30.853434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:30.854529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:30.855377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:30.855377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:30.855377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:30.855377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 251/509] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:30.855604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb0840-d545-4452-a244-bc4da75ee45c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-df905377-2cc7-48cb-8470-50340992e256 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:30.857552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:30.858414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:30.859315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:30.859315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:30.859315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:30.859315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:30.859315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:30.859315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:30.859315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:30.859315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:30.859315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:50:30.859315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:30.859315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 259/510] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:30.860303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df905377-2cc7-48cb-8470-50340992e256 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.918221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:30.942320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:30.961546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df905377-2cc7-48cb-8470-50340992e256 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.961885 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb0840-d545-4452-a244-bc4da75ee45c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.996470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e96cb4-ed78-43bc-bb34-e86f9f16dd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.996763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56be6d-7269-4d02-8827-303b0d8dbdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.997072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e96cb4-ed78-43bc-bb34-e86f9f16dd7b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.997356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e96cb4-ed78-43bc-bb34-e86f9f16dd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.997445 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56be6d-7269-4d02-8827-303b0d8dbdfe None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.997636 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e96cb4-ed78-43bc-bb34-e86f9f16dd7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.997736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56be6d-7269-4d02-8827-303b0d8dbdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.997906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56be6d-7269-4d02-8827-303b0d8dbdfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:31.065403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56be6d-7269-4d02-8827-303b0d8dbdfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:31.065768 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e96cb4-ed78-43bc-bb34-e86f9f16dd7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:31.066804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56be6d-7269-4d02-8827-303b0d8dbdfe None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:31.067204 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e96cb4-ed78-43bc-bb34-e86f9f16dd7b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:31.123434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:31.147355 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:31.166392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 252/511] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:31.171130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 260/512] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:31.171285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e96cb4-ed78-43bc-bb34-e86f9f16dd7b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:31.171366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56be6d-7269-4d02-8827-303b0d8dbdfe None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:31.266269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56be6d-7269-4d02-8827-303b0d8dbdfe None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:31.272185 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e96cb4-ed78-43bc-bb34-e86f9f16dd7b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:31.299673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce0c2d-e7e3-461d-a29d-1902ee804916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:31.300298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce0c2d-e7e3-461d-a29d-1902ee804916 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:31.300576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce0c2d-e7e3-461d-a29d-1902ee804916 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:31.300870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce0c2d-e7e3-461d-a29d-1902ee804916 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:31.301502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce0c2d-e7e3-461d-a29d-1902ee804916 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:31.302026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce0c2d-e7e3-461d-a29d-1902ee804916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:31.303156 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce0c2d-e7e3-461d-a29d-1902ee804916 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:31.307621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdb93b-b580-42e0-ba92-4d66cba41913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:31.308268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdb93b-b580-42e0-ba92-4d66cba41913 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:31.308550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdb93b-b580-42e0-ba92-4d66cba41913 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:31.308824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdb93b-b580-42e0-ba92-4d66cba41913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:31.314550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 261/513] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:31.315326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce0c2d-e7e3-461d-a29d-1902ee804916 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:31.323354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdb93b-b580-42e0-ba92-4d66cba41913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:31.324796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdb93b-b580-42e0-ba92-4d66cba41913 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b4bdb93b-b580-42e0-ba92-4d66cba41913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:31.339658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:31.340787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:31.341777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:31.341777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:31.341777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:31.341777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 253/514] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:31.341777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdb93b-b580-42e0-ba92-4d66cba41913 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:31.354303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:31.363355 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:31.420668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce0c2d-e7e3-461d-a29d-1902ee804916 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:31.445536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdb93b-b580-42e0-ba92-4d66cba41913 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:31.454351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bb9fb-4636-465f-987c-14ec558f912f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:31.454969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bb9fb-4636-465f-987c-14ec558f912f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:31.455248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bb9fb-4636-465f-987c-14ec558f912f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:31.455547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bb9fb-4636-465f-987c-14ec558f912f None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:31.456302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bb9fb-4636-465f-987c-14ec558f912f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:31.456613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bb9fb-4636-465f-987c-14ec558f912f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:31.458184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bb9fb-4636-465f-987c-14ec558f912f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:31.475053 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 262/515] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Aug 24 09:50:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:31.477148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bb9fb-4636-465f-987c-14ec558f912f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:31.479663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f439759-5eff-4674-bd1b-b9b79dcfcae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:31.480325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f439759-5eff-4674-bd1b-b9b79dcfcae2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:31.480604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f439759-5eff-4674-bd1b-b9b79dcfcae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:31.480881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f439759-5eff-4674-bd1b-b9b79dcfcae2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:31.491530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f439759-5eff-4674-bd1b-b9b79dcfcae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:31.493580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f439759-5eff-4674-bd1b-b9b79dcfcae2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0f439759-5eff-4674-bd1b-b9b79dcfcae2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:31.507594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:31.508630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:31.509463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:31.509463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:31.509463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:31.509463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 254/516] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:31.509687 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f439759-5eff-4674-bd1b-b9b79dcfcae2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:31.569963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:31.579597 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:31.582732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bb9fb-4636-465f-987c-14ec558f912f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:31.608829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f439759-5eff-4674-bd1b-b9b79dcfcae2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:31.618253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc55697-0397-4235-9771-2a1b85b799e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:31.618877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc55697-0397-4235-9771-2a1b85b799e5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:31.619157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc55697-0397-4235-9771-2a1b85b799e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:31.619432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc55697-0397-4235-9771-2a1b85b799e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:31.629536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc55697-0397-4235-9771-2a1b85b799e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:31.631005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc55697-0397-4235-9771-2a1b85b799e5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:31.641633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde40a77-d1fb-4d50-8c98-df889a03eadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:31.642222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40a77-d1fb-4d50-8c98-df889a03eadb None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:31.642501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40a77-d1fb-4d50-8c98-df889a03eadb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:31.642775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40a77-d1fb-4d50-8c98-df889a03eadb None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:31.643428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde40a77-d1fb-4d50-8c98-df889a03eadb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:31.644853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde40a77-d1fb-4d50-8c98-df889a03eadb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-4dc55697-0397-4235-9771-2a1b85b799e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:31.645019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:31.646006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:31.646859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:31.646859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:31.646859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:31.646859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 263/517] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:31.646859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc55697-0397-4235-9771-2a1b85b799e5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:31.667175 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 255/518] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:50:31 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:31.668006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde40a77-d1fb-4d50-8c98-df889a03eadb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:31.776854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc55697-0397-4235-9771-2a1b85b799e5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:31.786336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde40a77-d1fb-4d50-8c98-df889a03eadb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:31.806167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:31.819802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d05082-a4ef-4f48-8d42-eff5ed9abe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:31.820235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05082-a4ef-4f48-8d42-eff5ed9abe58 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:31.820547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05082-a4ef-4f48-8d42-eff5ed9abe58 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:31.820683 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05082-a4ef-4f48-8d42-eff5ed9abe58 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:31.821399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d05082-a4ef-4f48-8d42-eff5ed9abe58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:31.821701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d05082-a4ef-4f48-8d42-eff5ed9abe58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:31.823138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d05082-a4ef-4f48-8d42-eff5ed9abe58 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:31.829933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02dcf9-525c-4e70-9dc2-7a31ffeaef87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:31.830472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02dcf9-525c-4e70-9dc2-7a31ffeaef87 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:31.830811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02dcf9-525c-4e70-9dc2-7a31ffeaef87 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:31.831167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02dcf9-525c-4e70-9dc2-7a31ffeaef87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:31.839165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 264/519] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:31.839733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d05082-a4ef-4f48-8d42-eff5ed9abe58 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:31.855950 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:31.919268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02dcf9-525c-4e70-9dc2-7a31ffeaef87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:31.920767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02dcf9-525c-4e70-9dc2-7a31ffeaef87 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:31.949189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d05082-a4ef-4f48-8d42-eff5ed9abe58 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:31.983540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d03d7-cdf7-4734-944e-c0f50a23f0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:31.984218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d03d7-cdf7-4734-944e-c0f50a23f0bc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:31.984665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d03d7-cdf7-4734-944e-c0f50a23f0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:31.984947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d03d7-cdf7-4734-944e-c0f50a23f0bc None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:31.985539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d03d7-cdf7-4734-944e-c0f50a23f0bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:31.986072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d03d7-cdf7-4734-944e-c0f50a23f0bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:31.987136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d03d7-cdf7-4734-944e-c0f50a23f0bc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:31.998547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 265/520] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:31.999360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d03d7-cdf7-4734-944e-c0f50a23f0bc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.035171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 256/521] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:32.036369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02dcf9-525c-4e70-9dc2-7a31ffeaef87 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.055290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:32.076896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:32.109247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d03d7-cdf7-4734-944e-c0f50a23f0bc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.142045 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02dcf9-525c-4e70-9dc2-7a31ffeaef87 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.143831 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07bad7-adda-402d-a636-df22e003e816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:32.144467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07bad7-adda-402d-a636-df22e003e816 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.144743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07bad7-adda-402d-a636-df22e003e816 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.145010 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07bad7-adda-402d-a636-df22e003e816 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.145653 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07bad7-adda-402d-a636-df22e003e816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:32.145959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07bad7-adda-402d-a636-df22e003e816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:32.147323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07bad7-adda-402d-a636-df22e003e816 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:32.162456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 266/522] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:32.162456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07bad7-adda-402d-a636-df22e003e816 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.185235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813f564-7714-49e1-aa31-494da41eed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:32.185865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813f564-7714-49e1-aa31-494da41eed27 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.186147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813f564-7714-49e1-aa31-494da41eed27 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.186424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813f564-7714-49e1-aa31-494da41eed27 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.194031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813f564-7714-49e1-aa31-494da41eed27 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:32.196014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813f564-7714-49e1-aa31-494da41eed27 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-9813f564-7714-49e1-aa31-494da41eed27 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:32.207917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:32.208870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:32.209765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:32.209765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:32.209765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:32.209765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:32.209765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:32.209765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:32.209765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:32.209765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:32.209765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:50:32.209765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:32.209765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 257/523] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:32.210343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813f564-7714-49e1-aa31-494da41eed27 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.280131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07bad7-adda-402d-a636-df22e003e816 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.296115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:32.312383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:32.318351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae21cc-7247-43bd-b429-b4eb47edcda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:32.318971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae21cc-7247-43bd-b429-b4eb47edcda0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.319247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae21cc-7247-43bd-b429-b4eb47edcda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.319506 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae21cc-7247-43bd-b429-b4eb47edcda0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.322820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813f564-7714-49e1-aa31-494da41eed27 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.326484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae21cc-7247-43bd-b429-b4eb47edcda0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:32.327927 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae21cc-7247-43bd-b429-b4eb47edcda0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-e4ae21cc-7247-43bd-b429-b4eb47edcda0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:32.340232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:32.341109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:32.342099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:32.342099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:32.342099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:32.342099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:32.342099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:32.342099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:32.342099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:32.342099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:32.342099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 267/524] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:32.342099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae21cc-7247-43bd-b429-b4eb47edcda0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.368038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d042d-4227-4b1c-a9b5-e4ad652f43cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:32.368810 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d042d-4227-4b1c-a9b5-e4ad652f43cf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.369198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d042d-4227-4b1c-a9b5-e4ad652f43cf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.369726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d042d-4227-4b1c-a9b5-e4ad652f43cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.382770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d042d-4227-4b1c-a9b5-e4ad652f43cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:32.384382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d042d-4227-4b1c-a9b5-e4ad652f43cf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-806d042d-4227-4b1c-a9b5-e4ad652f43cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:32.399200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:32.400183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:32.401051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:32.401051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:32.401051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:32.401051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 258/525] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:32.401264 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d042d-4227-4b1c-a9b5-e4ad652f43cf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.451409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae21cc-7247-43bd-b429-b4eb47edcda0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.489951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20f47f-ee76-4a82-a0fb-900f69adbfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:32.490604 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f47f-ee76-4a82-a0fb-900f69adbfec None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.490837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f47f-ee76-4a82-a0fb-900f69adbfec None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.491084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f47f-ee76-4a82-a0fb-900f69adbfec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.497908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20f47f-ee76-4a82-a0fb-900f69adbfec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:32.499374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20f47f-ee76-4a82-a0fb-900f69adbfec None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:32.508505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d042d-4227-4b1c-a9b5-e4ad652f43cf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ed20f47f-ee76-4a82-a0fb-900f69adbfec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:32.511341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:32.512192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:32.513212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:32.513212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:32.513212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:32.513212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:32.513212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:32.513212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:32.513212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:32.513212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:32.517466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 268/526] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:32.518304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20f47f-ee76-4a82-a0fb-900f69adbfec None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.542308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:32.550494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b824cda-c499-4692-85c2-03531e8cd5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:32.551093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b824cda-c499-4692-85c2-03531e8cd5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.551515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b824cda-c499-4692-85c2-03531e8cd5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.551750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b824cda-c499-4692-85c2-03531e8cd5f1 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.552615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b824cda-c499-4692-85c2-03531e8cd5f1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:32.552984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b824cda-c499-4692-85c2-03531e8cd5f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:32.554366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b824cda-c499-4692-85c2-03531e8cd5f1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:32.573142 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:32.579358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 259/527] 10.222.0.86 () {56 vars in 1139 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:32.579358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b824cda-c499-4692-85c2-03531e8cd5f1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.634070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20f47f-ee76-4a82-a0fb-900f69adbfec None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.668220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d28e3-02a3-44bb-98aa-9bb9dd086f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:32.669095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d28e3-02a3-44bb-98aa-9bb9dd086f9c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.669464 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d28e3-02a3-44bb-98aa-9bb9dd086f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.669817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d28e3-02a3-44bb-98aa-9bb9dd086f9c None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.670518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d28e3-02a3-44bb-98aa-9bb9dd086f9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:32.670827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d28e3-02a3-44bb-98aa-9bb9dd086f9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:32.672674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d28e3-02a3-44bb-98aa-9bb9dd086f9c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:32.690000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b824cda-c499-4692-85c2-03531e8cd5f1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.695528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 269/528] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:32.695528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d28e3-02a3-44bb-98aa-9bb9dd086f9c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.730580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369dde3-73a0-4a1e-b654-b768317309d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:32.731189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369dde3-73a0-4a1e-b654-b768317309d5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.731467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369dde3-73a0-4a1e-b654-b768317309d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.731742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369dde3-73a0-4a1e-b654-b768317309d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.732418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369dde3-73a0-4a1e-b654-b768317309d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:32.732728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369dde3-73a0-4a1e-b654-b768317309d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:32.734098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369dde3-73a0-4a1e-b654-b768317309d5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:32.746856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 260/529] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:32.749522 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369dde3-73a0-4a1e-b654-b768317309d5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.802586 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d28e3-02a3-44bb-98aa-9bb9dd086f9c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.812327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:32.817265 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:32.841617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe1dbf-2c16-4c66-9166-8fc349016ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:32.842223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe1dbf-2c16-4c66-9166-8fc349016ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.842502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe1dbf-2c16-4c66-9166-8fc349016ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.842777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe1dbf-2c16-4c66-9166-8fc349016ce0 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.843470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe1dbf-2c16-4c66-9166-8fc349016ce0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:32.843903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe1dbf-2c16-4c66-9166-8fc349016ce0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:32.845284 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe1dbf-2c16-4c66-9166-8fc349016ce0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:32.857277 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 270/530] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:32.858251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe1dbf-2c16-4c66-9166-8fc349016ce0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.876647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369dde3-73a0-4a1e-b654-b768317309d5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.911134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611434d4-848d-4bd7-a62b-50f669dfddd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:32.911707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611434d4-848d-4bd7-a62b-50f669dfddd3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.911992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611434d4-848d-4bd7-a62b-50f669dfddd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.912314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611434d4-848d-4bd7-a62b-50f669dfddd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.964419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe1dbf-2c16-4c66-9166-8fc349016ce0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.991383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611434d4-848d-4bd7-a62b-50f669dfddd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:32.993456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611434d4-848d-4bd7-a62b-50f669dfddd3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:33.001121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e33cc-752d-4e3c-88c3-96ddb5b73793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:33.001775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e33cc-752d-4e3c-88c3-96ddb5b73793 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.002062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e33cc-752d-4e3c-88c3-96ddb5b73793 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.002339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e33cc-752d-4e3c-88c3-96ddb5b73793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.003095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e33cc-752d-4e3c-88c3-96ddb5b73793 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=794dbf38682e4b4f939854eecbacaf07) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:33.003407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e33cc-752d-4e3c-88c3-96ddb5b73793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:33.004876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e33cc-752d-4e3c-88c3-96ddb5b73793 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:33.010349 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 271/531] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/role_assignments?user.id=794dbf38682e4b4f939854eecbacaf07&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:33.011122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e33cc-752d-4e3c-88c3-96ddb5b73793 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:33.057641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:33.075727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:33.134551 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e33cc-752d-4e3c-88c3-96ddb5b73793 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:33.146860 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 261/532] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:33.148169 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611434d4-848d-4bd7-a62b-50f669dfddd3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:33.176665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008d7ac-c09f-4989-a483-4ba105092fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:33.177481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008d7ac-c09f-4989-a483-4ba105092fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.177481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008d7ac-c09f-4989-a483-4ba105092fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.177760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008d7ac-c09f-4989-a483-4ba105092fd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.186031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008d7ac-c09f-4989-a483-4ba105092fd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:33.187508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008d7ac-c09f-4989-a483-4ba105092fd5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-8008d7ac-c09f-4989-a483-4ba105092fd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:33.200574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:33.201451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:33.202315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:33.202315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:33.202315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:33.202315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:33.202315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:33.202315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:33.202315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:33.202315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:33.208200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 272/533] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:33.209113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008d7ac-c09f-4989-a483-4ba105092fd5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:33.261278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611434d4-848d-4bd7-a62b-50f669dfddd3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:33.299358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bed42-bf28-495d-b473-3e3842f0761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:33.300840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bed42-bf28-495d-b473-3e3842f0761b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.301597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bed42-bf28-495d-b473-3e3842f0761b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.302063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bed42-bf28-495d-b473-3e3842f0761b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.303237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bed42-bf28-495d-b473-3e3842f0761b None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=98e11312493643b199fca04cb2d77853) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:33.304132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bed42-bf28-495d-b473-3e3842f0761b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:33.305868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bed42-bf28-495d-b473-3e3842f0761b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:33.313822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:33.315731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 262/534] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:33 2022] GET /identity/v3/role_assignments?user.id=98e11312493643b199fca04cb2d77853&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:33.317347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bed42-bf28-495d-b473-3e3842f0761b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:33.322053 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008d7ac-c09f-4989-a483-4ba105092fd5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:33.339208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:33.377279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da57ec-174f-4f71-baed-3416f4bf857f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:33.377932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da57ec-174f-4f71-baed-3416f4bf857f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.378213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da57ec-174f-4f71-baed-3416f4bf857f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.378488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da57ec-174f-4f71-baed-3416f4bf857f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.406630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da57ec-174f-4f71-baed-3416f4bf857f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:33.408288 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da57ec-174f-4f71-baed-3416f4bf857f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-87da57ec-174f-4f71-baed-3416f4bf857f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:33.437318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:33.438503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:33.439348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:33.439348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:33.439348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:33.439348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 273/535] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:33.456602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da57ec-174f-4f71-baed-3416f4bf857f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:33.458762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bed42-bf28-495d-b473-3e3842f0761b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:33.501561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31dcea-9733-428e-af5d-65c21f452d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:33.502808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31dcea-9733-428e-af5d-65c21f452d07 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.503721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31dcea-9733-428e-af5d-65c21f452d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.504512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31dcea-9733-428e-af5d-65c21f452d07 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.505659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31dcea-9733-428e-af5d-65c21f452d07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:33.506565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31dcea-9733-428e-af5d-65c21f452d07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:33.509277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31dcea-9733-428e-af5d-65c21f452d07 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:33.538285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 263/536] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:33.539461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31dcea-9733-428e-af5d-65c21f452d07 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:33.611580 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da57ec-174f-4f71-baed-3416f4bf857f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:33.623192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:33.650865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d8890-2483-40a3-bb88-04a10154e767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:33.651645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d8890-2483-40a3-bb88-04a10154e767 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.652082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d8890-2483-40a3-bb88-04a10154e767 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.652512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d8890-2483-40a3-bb88-04a10154e767 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.653219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d8890-2483-40a3-bb88-04a10154e767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:33.653736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d8890-2483-40a3-bb88-04a10154e767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:33.655097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d8890-2483-40a3-bb88-04a10154e767 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:33.662829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31dcea-9733-428e-af5d-65c21f452d07 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:33.668981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 274/537] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:33.675016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e656f-8e32-44f3-8681-27c69ad8d535 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.675016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e656f-8e32-44f3-8681-27c69ad8d535 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.675016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e656f-8e32-44f3-8681-27c69ad8d535 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.675016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 275/538] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:33.676376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e656f-8e32-44f3-8681-27c69ad8d535 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:33.696554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:33.715142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef720de-082e-427c-97d1-cdb008ff6344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:33.716176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef720de-082e-427c-97d1-cdb008ff6344 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.716696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef720de-082e-427c-97d1-cdb008ff6344 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.717150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef720de-082e-427c-97d1-cdb008ff6344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.718285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef720de-082e-427c-97d1-cdb008ff6344 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=bd7b580705f247699aeb58772e668730) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:33.718743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef720de-082e-427c-97d1-cdb008ff6344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:33.721298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef720de-082e-427c-97d1-cdb008ff6344 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:33.728379 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 264/539] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:33 2022] GET /identity/v3/role_assignments?user.id=bd7b580705f247699aeb58772e668730&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:33.730312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe10f7e-3c47-48be-b0f6-8c223ae10429 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.730776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe10f7e-3c47-48be-b0f6-8c223ae10429 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.731213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe10f7e-3c47-48be-b0f6-8c223ae10429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.752822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-efe10f7e-3c47-48be-b0f6-8c223ae10429 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:33.754341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-efe10f7e-3c47-48be-b0f6-8c223ae10429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:33.848303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 265/540] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:33.857860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e656f-8e32-44f3-8681-27c69ad8d535 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:33.896357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c098e-5a14-4ffb-a06d-f2f5b795f70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:33.897001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c098e-5a14-4ffb-a06d-f2f5b795f70e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.897276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c098e-5a14-4ffb-a06d-f2f5b795f70e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.897542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c098e-5a14-4ffb-a06d-f2f5b795f70e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.905109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c098e-5a14-4ffb-a06d-f2f5b795f70e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:33.906782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c098e-5a14-4ffb-a06d-f2f5b795f70e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c32c098e-5a14-4ffb-a06d-f2f5b795f70e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:33.919089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:33.920003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:33.921152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:33.921152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:33.921152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:33.921152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:33.921152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:33.921152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:33.921152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:33.921152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:33.921152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 276/541] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:33.926561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe10f7e-3c47-48be-b0f6-8c223ae10429 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:34.004580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:34.053228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe10f7e-3c47-48be-b0f6-8c223ae10429 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:34.089312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e047c5-4e70-4ca2-8817-d299d2de334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:34.090274 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e047c5-4e70-4ca2-8817-d299d2de334d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:34.090753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e047c5-4e70-4ca2-8817-d299d2de334d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:34.091252 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e047c5-4e70-4ca2-8817-d299d2de334d None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:34.092183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e047c5-4e70-4ca2-8817-d299d2de334d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:34.092703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e047c5-4e70-4ca2-8817-d299d2de334d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:34.094373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e047c5-4e70-4ca2-8817-d299d2de334d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:34.106815 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 266/542] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:34.112442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c098e-5a14-4ffb-a06d-f2f5b795f70e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:34.201072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:34.225120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e047c5-4e70-4ca2-8817-d299d2de334d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:34.238476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c098e-5a14-4ffb-a06d-f2f5b795f70e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:34.273183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dc77c-cefe-43dc-bd82-67a62b536fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:34.273991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc77c-cefe-43dc-bd82-67a62b536fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:34.273991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc77c-cefe-43dc-bd82-67a62b536fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:34.274271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc77c-cefe-43dc-bd82-67a62b536fc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:34.275005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dc77c-cefe-43dc-bd82-67a62b536fc2 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=27cc091f3b094cb1a1e2650a3fbd7671) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:34.275322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dc77c-cefe-43dc-bd82-67a62b536fc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:34.277247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dc77c-cefe-43dc-bd82-67a62b536fc2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:34.284161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 277/543] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:34 2022] GET /identity/v3/role_assignments?user.id=27cc091f3b094cb1a1e2650a3fbd7671&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:34.435157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e047c5-4e70-4ca2-8817-d299d2de334d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:34.460957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:34.470193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1769ab-7126-4060-ae49-a7bd5122d2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:34.470812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1769ab-7126-4060-ae49-a7bd5122d2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:34.471089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1769ab-7126-4060-ae49-a7bd5122d2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:34.471360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1769ab-7126-4060-ae49-a7bd5122d2d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:34.485762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1769ab-7126-4060-ae49-a7bd5122d2d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-de1769ab-7126-4060-ae49-a7bd5122d2d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:34.491210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:34.492490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:34.493330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:34.496634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 267/544] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:34.502750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dc77c-cefe-43dc-bd82-67a62b536fc2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:34.651073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dc77c-cefe-43dc-bd82-67a62b536fc2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:34.691643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:34.697442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991829a-1fab-4bbc-bf75-fca166e66c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:34.698235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991829a-1fab-4bbc-bf75-fca166e66c93 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:34.698719 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991829a-1fab-4bbc-bf75-fca166e66c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:34.698995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991829a-1fab-4bbc-bf75-fca166e66c93 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:34.699658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991829a-1fab-4bbc-bf75-fca166e66c93 None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:34.699961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991829a-1fab-4bbc-bf75-fca166e66c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:34.701442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991829a-1fab-4bbc-bf75-fca166e66c93 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:34.714087 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 278/545] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:34.719752 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1769ab-7126-4060-ae49-a7bd5122d2d8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:34.891805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1769ab-7126-4060-ae49-a7bd5122d2d8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:34.943116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd15d5-7c12-440d-831a-ce8edae55248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:34.943747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd15d5-7c12-440d-831a-ce8edae55248 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:34.944021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd15d5-7c12-440d-831a-ce8edae55248 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:34.944314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd15d5-7c12-440d-831a-ce8edae55248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:34.959671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd15d5-7c12-440d-831a-ce8edae55248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:34.961166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd15d5-7c12-440d-831a-ce8edae55248 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-16cd15d5-7c12-440d-831a-ce8edae55248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:34.980362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:34.981291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:34.982552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:34.982552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:34.982552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:34.982552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 268/546] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:34.988776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991829a-1fab-4bbc-bf75-fca166e66c93 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:35.141865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991829a-1fab-4bbc-bf75-fca166e66c93 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:35.188227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9433ba4-588b-41af-826c-208988771842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:35.188901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9433ba4-588b-41af-826c-208988771842 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:35.189176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9433ba4-588b-41af-826c-208988771842 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:35.189446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9433ba4-588b-41af-826c-208988771842 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:35.190081 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9433ba4-588b-41af-826c-208988771842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:35.190380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9433ba4-588b-41af-826c-208988771842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:35.191750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9433ba4-588b-41af-826c-208988771842 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:35.207362 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 279/547] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:35.213539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd15d5-7c12-440d-831a-ce8edae55248 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:35.292446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:35.384081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd15d5-7c12-440d-831a-ce8edae55248 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:35.409890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc40a6-e44e-47e0-97c5-8f774a14ef1d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:35.410190 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc40a6-e44e-47e0-97c5-8f774a14ef1d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:35.410451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc40a6-e44e-47e0-97c5-8f774a14ef1d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:35.412028 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 280/548] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:35.417187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ccffb-732a-4926-b383-557a10e10132 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:35.417457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ccffb-732a-4926-b383-557a10e10132 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:35.417733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ccffb-732a-4926-b383-557a10e10132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:35.423972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec97557-eec9-4ab8-bf3d-000c90bf0e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:35.424901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec97557-eec9-4ab8-bf3d-000c90bf0e87 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:35.425349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec97557-eec9-4ab8-bf3d-000c90bf0e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:35.425766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec97557-eec9-4ab8-bf3d-000c90bf0e87 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:35.434514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec97557-eec9-4ab8-bf3d-000c90bf0e87 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:35.436183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec97557-eec9-4ab8-bf3d-000c90bf0e87 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4ec97557-eec9-4ab8-bf3d-000c90bf0e87 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:35.448650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:35.449763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:35.451539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:35.451539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:35.451539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:35.451539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:35.451539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:35.451539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:35.451539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:35.451539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:35.451539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:50:35.451539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:35.453746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 269/549] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:35.461944 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-df8ccffb-732a-4926-b383-557a10e10132 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:35.461944 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-df8ccffb-732a-4926-b383-557a10e10132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:35.463247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec97557-eec9-4ab8-bf3d-000c90bf0e87 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:35.540794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:35.573314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 281/550] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:35.613088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec97557-eec9-4ab8-bf3d-000c90bf0e87 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:35.624430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:35.657601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b7592-505b-40dd-8907-68cbd942f722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:35.658555 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b7592-505b-40dd-8907-68cbd942f722 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:35.658975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b7592-505b-40dd-8907-68cbd942f722 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:35.659387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b7592-505b-40dd-8907-68cbd942f722 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:35.660341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b7592-505b-40dd-8907-68cbd942f722 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:35.660797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b7592-505b-40dd-8907-68cbd942f722 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:35.662454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b7592-505b-40dd-8907-68cbd942f722 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:35.689558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 270/551] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Aug 24 09:50:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:35.696537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ccffb-732a-4926-b383-557a10e10132 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:35.824438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffd2a9-fa45-4e85-b17d-10b74dc53b1c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:35.824957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffd2a9-fa45-4e85-b17d-10b74dc53b1c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:35.825046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffd2a9-fa45-4e85-b17d-10b74dc53b1c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:35.826566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 271/552] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:35.831904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fa283-7140-4fbd-b70a-4cbb3d7ce1df None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:35.832232 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fa283-7140-4fbd-b70a-4cbb3d7ce1df None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:35.832848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fa283-7140-4fbd-b70a-4cbb3d7ce1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:35.855324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-530fa283-7140-4fbd-b70a-4cbb3d7ce1df None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:35.856738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-530fa283-7140-4fbd-b70a-4cbb3d7ce1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:35.868357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ccffb-732a-4926-b383-557a10e10132 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:35.935298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb9fdb-20a3-424f-a980-4a3af1f36d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:35.935900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb9fdb-20a3-424f-a980-4a3af1f36d13 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:35.936199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb9fdb-20a3-424f-a980-4a3af1f36d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:35.936469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb9fdb-20a3-424f-a980-4a3af1f36d13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:35.965726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 272/553] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:35.966590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fa283-7140-4fbd-b70a-4cbb3d7ce1df None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:36.005067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:36.026356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb9fdb-20a3-424f-a980-4a3af1f36d13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:36.027710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb9fdb-20a3-424f-a980-4a3af1f36d13 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:36.117515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:36.142494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fa283-7140-4fbd-b70a-4cbb3d7ce1df None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:36.221739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7dc19-366f-44ba-9866-28fa19ae526e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:36.222380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7dc19-366f-44ba-9866-28fa19ae526e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:36.222655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7dc19-366f-44ba-9866-28fa19ae526e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:36.222969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7dc19-366f-44ba-9866-28fa19ae526e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:36.235716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 282/554] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:36.235716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc37818-f302-48f9-91c1-466136992148 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:36.235716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc37818-f302-48f9-91c1-466136992148 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:36.235716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc37818-f302-48f9-91c1-466136992148 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:36.235716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 283/555] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:36.240455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc37818-f302-48f9-91c1-466136992148 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:36.243591 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7dc19-366f-44ba-9866-28fa19ae526e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-cfe7dc19-366f-44ba-9866-28fa19ae526e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:36.248775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:36.249859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:36.250674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:36.253599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 273/556] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:36.254831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7a8cb-69d6-4d7c-b6b9-fa8daa24369d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:36.255120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7a8cb-69d6-4d7c-b6b9-fa8daa24369d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:36.255396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7a8cb-69d6-4d7c-b6b9-fa8daa24369d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:36.274989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-eab7a8cb-69d6-4d7c-b6b9-fa8daa24369d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:36.276254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eab7a8cb-69d6-4d7c-b6b9-fa8daa24369d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:36.342292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:36.350851 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 274/557] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:36.351991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7a8cb-69d6-4d7c-b6b9-fa8daa24369d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:36.365336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc37818-f302-48f9-91c1-466136992148 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:36.410221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41590626-0541-4a2a-944b-9a4bc722f73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:36.411428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590626-0541-4a2a-944b-9a4bc722f73e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:36.412300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590626-0541-4a2a-944b-9a4bc722f73e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:36.413079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590626-0541-4a2a-944b-9a4bc722f73e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:36.427306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41590626-0541-4a2a-944b-9a4bc722f73e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:36.429653 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41590626-0541-4a2a-944b-9a4bc722f73e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:36.448019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-41590626-0541-4a2a-944b-9a4bc722f73e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:36.449889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:36.451523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:36.452773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:36.452773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:36.452773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:36.454868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 284/558] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:36.455898 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41590626-0541-4a2a-944b-9a4bc722f73e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:36.484008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7a8cb-69d6-4d7c-b6b9-fa8daa24369d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:36.534387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca04c6-fb4a-461e-baa0-b2a462c6e19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:36.535421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca04c6-fb4a-461e-baa0-b2a462c6e19c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:36.535709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca04c6-fb4a-461e-baa0-b2a462c6e19c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:36.535986 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca04c6-fb4a-461e-baa0-b2a462c6e19c None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:36.536976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca04c6-fb4a-461e-baa0-b2a462c6e19c None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:36.537289 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca04c6-fb4a-461e-baa0-b2a462c6e19c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:36.538950 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca04c6-fb4a-461e-baa0-b2a462c6e19c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:36.551507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 275/559] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:36.552494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca04c6-fb4a-461e-baa0-b2a462c6e19c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:36.593428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41590626-0541-4a2a-944b-9a4bc722f73e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:36.639545 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c17aee-5aa2-421b-93ba-dfa0cb58d221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:36.640220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c17aee-5aa2-421b-93ba-dfa0cb58d221 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:36.640500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c17aee-5aa2-421b-93ba-dfa0cb58d221 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:36.640770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c17aee-5aa2-421b-93ba-dfa0cb58d221 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:36.643562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:36.654314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c17aee-5aa2-421b-93ba-dfa0cb58d221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a5c17aee-5aa2-421b-93ba-dfa0cb58d221 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:36.659929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:36.661383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 285/560] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:36.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c17aee-5aa2-421b-93ba-dfa0cb58d221 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:36.668825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca04c6-fb4a-461e-baa0-b2a462c6e19c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:36.703131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc58f7d-e4bc-44ef-a889-d8b3cf36d414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:36.703729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc58f7d-e4bc-44ef-a889-d8b3cf36d414 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:36.704006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc58f7d-e4bc-44ef-a889-d8b3cf36d414 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:36.704303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc58f7d-e4bc-44ef-a889-d8b3cf36d414 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:36.704950 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc58f7d-e4bc-44ef-a889-d8b3cf36d414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:36.705248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc58f7d-e4bc-44ef-a889-d8b3cf36d414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:36.706641 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc58f7d-e4bc-44ef-a889-d8b3cf36d414 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:36.713587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:36.719189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 276/561] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:36.720215 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc58f7d-e4bc-44ef-a889-d8b3cf36d414 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:36.761202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c17aee-5aa2-421b-93ba-dfa0cb58d221 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:36.796401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447929c-1939-414c-a710-b7b125a065e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:36.797024 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447929c-1939-414c-a710-b7b125a065e5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:36.797305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447929c-1939-414c-a710-b7b125a065e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:36.797583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447929c-1939-414c-a710-b7b125a065e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:36.808843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447929c-1939-414c-a710-b7b125a065e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:36.810287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447929c-1939-414c-a710-b7b125a065e5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:36.820743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc58f7d-e4bc-44ef-a889-d8b3cf36d414 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-9447929c-1939-414c-a710-b7b125a065e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:36.824602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:36.825649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:36.826558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:36.826558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:36.826558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:36.826558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 286/562] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:36.826558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447929c-1939-414c-a710-b7b125a065e5 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:36.856721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ad0fc-d8c7-4da8-bd52-1ae8d995f8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:36.857268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ad0fc-d8c7-4da8-bd52-1ae8d995f8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:36.857557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ad0fc-d8c7-4da8-bd52-1ae8d995f8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:36.857809 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ad0fc-d8c7-4da8-bd52-1ae8d995f8d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:36.868490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ad0fc-d8c7-4da8-bd52-1ae8d995f8d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e46ad0fc-d8c7-4da8-bd52-1ae8d995f8d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:36.873503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:36.874437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:36.875300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 277/563] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:36.876511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34fa07-29a8-49e2-8a4f-36e83cf49cd0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:36.876511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34fa07-29a8-49e2-8a4f-36e83cf49cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:36.876796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34fa07-29a8-49e2-8a4f-36e83cf49cd0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:36.877411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:36.878038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 278/564] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:36.880854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34fa07-29a8-49e2-8a4f-36e83cf49cd0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:36.927444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447929c-1939-414c-a710-b7b125a065e5 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:36.939307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:36.962906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de9a4a-39c9-4b25-987a-2fbef55733c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:36.963504 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de9a4a-39c9-4b25-987a-2fbef55733c1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:36.963781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de9a4a-39c9-4b25-987a-2fbef55733c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:36.964071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de9a4a-39c9-4b25-987a-2fbef55733c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:36.964735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de9a4a-39c9-4b25-987a-2fbef55733c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:36.965036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de9a4a-39c9-4b25-987a-2fbef55733c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:36.966389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de9a4a-39c9-4b25-987a-2fbef55733c1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:36.974332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 287/565] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:36.975175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de9a4a-39c9-4b25-987a-2fbef55733c1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:36.983754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34fa07-29a8-49e2-8a4f-36e83cf49cd0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:37.017989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fb649-030c-46b4-b689-8887531a75ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:37.018605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fb649-030c-46b4-b689-8887531a75ca None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.018880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fb649-030c-46b4-b689-8887531a75ca None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.019152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fb649-030c-46b4-b689-8887531a75ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.025809 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7fb649-030c-46b4-b689-8887531a75ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:37.027249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7fb649-030c-46b4-b689-8887531a75ca None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0e7fb649-030c-46b4-b689-8887531a75ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:37.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:37.039655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:37.040794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:37.040794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:37.040794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:37.040794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:37.040794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:37.040794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:37.040794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:37.040794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:37.040794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 279/566] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:37.040794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fb649-030c-46b4-b689-8887531a75ca None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:37.082130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de9a4a-39c9-4b25-987a-2fbef55733c1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:37.092263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:37.116318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f24b2d-e39f-440b-9700-28561e173593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:37.116840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f24b2d-e39f-440b-9700-28561e173593 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.117116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f24b2d-e39f-440b-9700-28561e173593 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.117386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f24b2d-e39f-440b-9700-28561e173593 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.118530 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f24b2d-e39f-440b-9700-28561e173593 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:37.118826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f24b2d-e39f-440b-9700-28561e173593 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:37.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f24b2d-e39f-440b-9700-28561e173593 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:37.131879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 288/567] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:37.132933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffef009-5e10-4138-8c7b-697e0e6ac2ae None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.133217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffef009-5e10-4138-8c7b-697e0e6ac2ae None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.133535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffef009-5e10-4138-8c7b-697e0e6ac2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.147793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fb649-030c-46b4-b689-8887531a75ca None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:37.150917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7ffef009-5e10-4138-8c7b-697e0e6ac2ae None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:37.152138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7ffef009-5e10-4138-8c7b-697e0e6ac2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:37.166119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:37.183804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e3b3f-b36d-46fb-9b88-e7f9f14c6e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:37.184442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e3b3f-b36d-46fb-9b88-e7f9f14c6e03 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.184725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e3b3f-b36d-46fb-9b88-e7f9f14c6e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.185005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e3b3f-b36d-46fb-9b88-e7f9f14c6e03 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.185663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e3b3f-b36d-46fb-9b88-e7f9f14c6e03 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:37.185967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e3b3f-b36d-46fb-9b88-e7f9f14c6e03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:37.187331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e3b3f-b36d-46fb-9b88-e7f9f14c6e03 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:37.195340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 280/568] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:37.196175 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e3b3f-b36d-46fb-9b88-e7f9f14c6e03 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:37.220330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 289/569] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:37.221171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffef009-5e10-4138-8c7b-697e0e6ac2ae None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:37.301184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e3b3f-b36d-46fb-9b88-e7f9f14c6e03 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:37.320777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:37.323776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffef009-5e10-4138-8c7b-697e0e6ac2ae None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:37.338532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d804d-5653-418c-bb81-8edcaa8e3041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:37.339151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d804d-5653-418c-bb81-8edcaa8e3041 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.339428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d804d-5653-418c-bb81-8edcaa8e3041 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.339698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d804d-5653-418c-bb81-8edcaa8e3041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.351558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d804d-5653-418c-bb81-8edcaa8e3041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:37.353813 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d804d-5653-418c-bb81-8edcaa8e3041 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:37.357998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed1622-6464-4441-8f95-a32c7f35a350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:37.358577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed1622-6464-4441-8f95-a32c7f35a350 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.358857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed1622-6464-4441-8f95-a32c7f35a350 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.359129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed1622-6464-4441-8f95-a32c7f35a350 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.359814 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed1622-6464-4441-8f95-a32c7f35a350 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:37.360352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed1622-6464-4441-8f95-a32c7f35a350 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:37.361724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed1622-6464-4441-8f95-a32c7f35a350 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e64d804d-5653-418c-bb81-8edcaa8e3041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:37.368512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:37.369676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:37.370727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:37.370727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:37.370727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:37.370727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 281/570] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:37.370727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d804d-5653-418c-bb81-8edcaa8e3041 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:37.384225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 290/571] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:37.384225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed1622-6464-4441-8f95-a32c7f35a350 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:37.411938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:37.487981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed1622-6464-4441-8f95-a32c7f35a350 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:37.488237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d804d-5653-418c-bb81-8edcaa8e3041 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:37.522153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46a546-3671-4e1a-a91f-36c8b84dcd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:37.522490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a9f05-0bdb-4def-b0ab-421c5d67ce96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:37.522951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46a546-3671-4e1a-a91f-36c8b84dcd8e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.523116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a9f05-0bdb-4def-b0ab-421c5d67ce96 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.523361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46a546-3671-4e1a-a91f-36c8b84dcd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.523449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a9f05-0bdb-4def-b0ab-421c5d67ce96 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.523659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a9f05-0bdb-4def-b0ab-421c5d67ce96 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.523803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46a546-3671-4e1a-a91f-36c8b84dcd8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.532620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a9f05-0bdb-4def-b0ab-421c5d67ce96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:37.534063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a9f05-0bdb-4def-b0ab-421c5d67ce96 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:37.535110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46a546-3671-4e1a-a91f-36c8b84dcd8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:37.536594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46a546-3671-4e1a-a91f-36c8b84dcd8e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-268a9f05-0bdb-4def-b0ab-421c5d67ce96 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:37.544773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:37.545953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:37.547144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:37.547144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:37.547144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:37.547144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:37.547144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:37.547144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:37.547144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:37.547144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:37.547144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:50:37.547144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:37.548422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 282/572] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:37.549425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a9f05-0bdb-4def-b0ab-421c5d67ce96 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ab46a546-3671-4e1a-a91f-36c8b84dcd8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:37.550710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:37.551804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:37.552769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:37.552769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:37.552769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:37.556885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 291/573] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:37.557589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46a546-3671-4e1a-a91f-36c8b84dcd8e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:37.572651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:37.640651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:37.649884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a9f05-0bdb-4def-b0ab-421c5d67ce96 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:37.656351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46a546-3671-4e1a-a91f-36c8b84dcd8e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:37.683579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cf938-35b9-433b-a756-5246f8307020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:37.684215 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cf938-35b9-433b-a756-5246f8307020 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.684503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cf938-35b9-433b-a756-5246f8307020 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.684781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cf938-35b9-433b-a756-5246f8307020 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.685421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cf938-35b9-433b-a756-5246f8307020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:37.685726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cf938-35b9-433b-a756-5246f8307020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:37.687108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cf938-35b9-433b-a756-5246f8307020 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:37.689869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ea13e-28ec-4b4d-823e-c0fabd584529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:37.690451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea13e-28ec-4b4d-823e-c0fabd584529 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.690726 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea13e-28ec-4b4d-823e-c0fabd584529 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.690992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea13e-28ec-4b4d-823e-c0fabd584529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.691700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ea13e-28ec-4b4d-823e-c0fabd584529 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=f1b1d85195db46b19d6a0156dbdf7f6e) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:37.692003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ea13e-28ec-4b4d-823e-c0fabd584529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:37.693447 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ea13e-28ec-4b4d-823e-c0fabd584529 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:37.699296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 283/574] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:37.705756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 292/575] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:37 2022] GET /identity/v3/role_assignments?user.id=f1b1d85195db46b19d6a0156dbdf7f6e&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:37.705994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cf938-35b9-433b-a756-5246f8307020 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:37.706093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ea13e-28ec-4b4d-823e-c0fabd584529 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:37.795399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:37.820907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cf938-35b9-433b-a756-5246f8307020 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:37.823632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ea13e-28ec-4b4d-823e-c0fabd584529 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:37.859209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39a701-deca-4372-8e3e-b04d826befcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:37.859950 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39a701-deca-4372-8e3e-b04d826befcd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.860282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39a701-deca-4372-8e3e-b04d826befcd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.860558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39a701-deca-4372-8e3e-b04d826befcd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.863196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea7c8f-6a48-46b2-90d7-a1f9deca15e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:37.863803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea7c8f-6a48-46b2-90d7-a1f9deca15e9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.864137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea7c8f-6a48-46b2-90d7-a1f9deca15e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.864660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea7c8f-6a48-46b2-90d7-a1f9deca15e9 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.865116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea7c8f-6a48-46b2-90d7-a1f9deca15e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:37.865421 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea7c8f-6a48-46b2-90d7-a1f9deca15e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:37.866848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea7c8f-6a48-46b2-90d7-a1f9deca15e9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:37.871877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39a701-deca-4372-8e3e-b04d826befcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8c39a701-deca-4372-8e3e-b04d826befcd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:37.876556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:37.877471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:37.878320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 284/576] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:37.880459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39a701-deca-4372-8e3e-b04d826befcd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:37.894753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:37.895337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 293/577] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Aug 24 09:50:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:37.896100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea7c8f-6a48-46b2-90d7-a1f9deca15e9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:37.986225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39a701-deca-4372-8e3e-b04d826befcd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.005582 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea7c8f-6a48-46b2-90d7-a1f9deca15e9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.021438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9233ca-88fa-42d8-a34f-de5777a38d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.022050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9233ca-88fa-42d8-a34f-de5777a38d81 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.022335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9233ca-88fa-42d8-a34f-de5777a38d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.022614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9233ca-88fa-42d8-a34f-de5777a38d81 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.023261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9233ca-88fa-42d8-a34f-de5777a38d81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:38.023579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9233ca-88fa-42d8-a34f-de5777a38d81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.025013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9233ca-88fa-42d8-a34f-de5777a38d81 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.036313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 285/578] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:38.037133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9233ca-88fa-42d8-a34f-de5777a38d81 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.048831 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eace95-e2c2-4491-b772-8ffe96f7667b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.049859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eace95-e2c2-4491-b772-8ffe96f7667b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.050143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eace95-e2c2-4491-b772-8ffe96f7667b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.050416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eace95-e2c2-4491-b772-8ffe96f7667b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.052569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:38.061062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eace95-e2c2-4491-b772-8ffe96f7667b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.062648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eace95-e2c2-4491-b772-8ffe96f7667b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-23eace95-e2c2-4491-b772-8ffe96f7667b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:38.075118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:38.076079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:38.077033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:38.077033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:38.077033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:38.077033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:38.077033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:38.077033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:38.077033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:38.077033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:38.077033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:50:38.077033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:38.077033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 294/579] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Aug 24 09:50:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:38.077618 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eace95-e2c2-4491-b772-8ffe96f7667b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.117868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:38.149322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9233ca-88fa-42d8-a34f-de5777a38d81 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.184800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b77698-5c09-4ab0-a0a2-7176a9bd1429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.185410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b77698-5c09-4ab0-a0a2-7176a9bd1429 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.185694 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b77698-5c09-4ab0-a0a2-7176a9bd1429 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.185968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b77698-5c09-4ab0-a0a2-7176a9bd1429 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.186515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eace95-e2c2-4491-b772-8ffe96f7667b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.186617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b77698-5c09-4ab0-a0a2-7176a9bd1429 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:38.186920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b77698-5c09-4ab0-a0a2-7176a9bd1429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.188326 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b77698-5c09-4ab0-a0a2-7176a9bd1429 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.196366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 286/580] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:38.196366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b77698-5c09-4ab0-a0a2-7176a9bd1429 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.226716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb756c2-3116-47ec-a8c8-8888b46034a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.227883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb756c2-3116-47ec-a8c8-8888b46034a2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.228221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb756c2-3116-47ec-a8c8-8888b46034a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.228500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb756c2-3116-47ec-a8c8-8888b46034a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.239609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb756c2-3116-47ec-a8c8-8888b46034a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.241070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb756c2-3116-47ec-a8c8-8888b46034a2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-bfb756c2-3116-47ec-a8c8-8888b46034a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:38.255875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:38.257071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:38.258121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:38.258121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:38.258121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:38.258121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 295/581] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:38.258671 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb756c2-3116-47ec-a8c8-8888b46034a2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.310996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:38.318766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b77698-5c09-4ab0-a0a2-7176a9bd1429 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.359208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46b5b2-38a4-4520-bd4a-a5136370f255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.359954 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46b5b2-38a4-4520-bd4a-a5136370f255 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.360262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46b5b2-38a4-4520-bd4a-a5136370f255 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.360541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46b5b2-38a4-4520-bd4a-a5136370f255 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.362579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:38.365712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb756c2-3116-47ec-a8c8-8888b46034a2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.370283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46b5b2-38a4-4520-bd4a-a5136370f255 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.371808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46b5b2-38a4-4520-bd4a-a5136370f255 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2e46b5b2-38a4-4520-bd4a-a5136370f255 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:38.384767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:38.385996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:38.385996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:38.385996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:38.385996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:38.385996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:38.385996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:38.385996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:38.385996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:38.385996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:50:38.385996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:38.391392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 287/582] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:38.391392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46b5b2-38a4-4520-bd4a-a5136370f255 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.406260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99801b99-99cf-46a7-b007-882ebf39376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.406909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99801b99-99cf-46a7-b007-882ebf39376c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.407255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99801b99-99cf-46a7-b007-882ebf39376c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.407781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99801b99-99cf-46a7-b007-882ebf39376c None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.408721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99801b99-99cf-46a7-b007-882ebf39376c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:38.409109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99801b99-99cf-46a7-b007-882ebf39376c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.410543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99801b99-99cf-46a7-b007-882ebf39376c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.428538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 296/583] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:38.429360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99801b99-99cf-46a7-b007-882ebf39376c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.503750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46b5b2-38a4-4520-bd4a-a5136370f255 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.534425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99801b99-99cf-46a7-b007-882ebf39376c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.536857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:38.544690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7e9a3-267a-4a7d-bc60-9f11824853fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.545470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7e9a3-267a-4a7d-bc60-9f11824853fc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.545821 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7e9a3-267a-4a7d-bc60-9f11824853fc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.546163 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7e9a3-267a-4a7d-bc60-9f11824853fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.557579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7e9a3-267a-4a7d-bc60-9f11824853fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.559305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7e9a3-267a-4a7d-bc60-9f11824853fc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.570869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0d241-5d74-49f2-a2b9-f5abeacdf908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.571571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0d241-5d74-49f2-a2b9-f5abeacdf908 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.571925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0d241-5d74-49f2-a2b9-f5abeacdf908 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.572292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0d241-5d74-49f2-a2b9-f5abeacdf908 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.573007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0d241-5d74-49f2-a2b9-f5abeacdf908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:38.573384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0d241-5d74-49f2-a2b9-f5abeacdf908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-efe7e9a3-267a-4a7d-bc60-9f11824853fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:38.574579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:38.575667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:38.576819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:38.576819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:38.576819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:38.577038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0d241-5d74-49f2-a2b9-f5abeacdf908 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.582338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 288/584] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:38.582338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7e9a3-267a-4a7d-bc60-9f11824853fc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.594443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 297/585] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:38.595398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0d241-5d74-49f2-a2b9-f5abeacdf908 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.633538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:38.693246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7e9a3-267a-4a7d-bc60-9f11824853fc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.704723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0d241-5d74-49f2-a2b9-f5abeacdf908 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.730986 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b9a49-6398-43ab-9278-b9e775db6110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.731686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b9a49-6398-43ab-9278-b9e775db6110 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.732062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b9a49-6398-43ab-9278-b9e775db6110 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.732433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b9a49-6398-43ab-9278-b9e775db6110 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.733230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b9a49-6398-43ab-9278-b9e775db6110 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:38.733606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b9a49-6398-43ab-9278-b9e775db6110 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.735111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b9a49-6398-43ab-9278-b9e775db6110 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.740941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e8ebe-cc15-4f8b-be51-55db4f8e2a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.741568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e8ebe-cc15-4f8b-be51-55db4f8e2a3e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.741854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e8ebe-cc15-4f8b-be51-55db4f8e2a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.742123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e8ebe-cc15-4f8b-be51-55db4f8e2a3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.753138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 289/586] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:38.754121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b9a49-6398-43ab-9278-b9e775db6110 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.759836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e8ebe-cc15-4f8b-be51-55db4f8e2a3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.761655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e8ebe-cc15-4f8b-be51-55db4f8e2a3e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-f79e8ebe-cc15-4f8b-be51-55db4f8e2a3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:38.777666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:38.778593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:38.779525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:38.779525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:38.779525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:38.779525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 298/587] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:38.779525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e8ebe-cc15-4f8b-be51-55db4f8e2a3e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.801457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:38.873407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b9a49-6398-43ab-9278-b9e775db6110 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.876746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:38.888383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e8ebe-cc15-4f8b-be51-55db4f8e2a3e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.910832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eebc6-d79e-4de9-86bb-88ef32755e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.911437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eebc6-d79e-4de9-86bb-88ef32755e10 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.911716 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eebc6-d79e-4de9-86bb-88ef32755e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.911990 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eebc6-d79e-4de9-86bb-88ef32755e10 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.912758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62eebc6-d79e-4de9-86bb-88ef32755e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:38.912961 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62eebc6-d79e-4de9-86bb-88ef32755e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.914326 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62eebc6-d79e-4de9-86bb-88ef32755e10 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.926036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d207dc-9370-494b-a276-caf50d9a2a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.930926 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 290/588] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:38.931050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d207dc-9370-494b-a276-caf50d9a2a42 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.931050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d207dc-9370-494b-a276-caf50d9a2a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.931050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d207dc-9370-494b-a276-caf50d9a2a42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.931261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eebc6-d79e-4de9-86bb-88ef32755e10 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.938102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d207dc-9370-494b-a276-caf50d9a2a42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.939561 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d207dc-9370-494b-a276-caf50d9a2a42 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-15d207dc-9370-494b-a276-caf50d9a2a42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:38.952763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:38.953736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:38.954646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:38.954646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:38.954646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:38.954646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:38.954646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:38.954646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:38.954646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:38.954646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:38.954646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 299/589] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:38.954646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d207dc-9370-494b-a276-caf50d9a2a42 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:39.036147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:39.043444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eebc6-d79e-4de9-86bb-88ef32755e10 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:39.062524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d207dc-9370-494b-a276-caf50d9a2a42 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:39.078111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a6f58-7149-47d8-a674-abe9ae430455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:39.078734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a6f58-7149-47d8-a674-abe9ae430455 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.079013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a6f58-7149-47d8-a674-abe9ae430455 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.079284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a6f58-7149-47d8-a674-abe9ae430455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.090496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a6f58-7149-47d8-a674-abe9ae430455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:39.091955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a6f58-7149-47d8-a674-abe9ae430455 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:39.099105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f41ceb-b632-4f59-bf0d-682cc6122188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:39.099773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41ceb-b632-4f59-bf0d-682cc6122188 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.100149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41ceb-b632-4f59-bf0d-682cc6122188 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.100486 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41ceb-b632-4f59-bf0d-682cc6122188 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.101126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41ceb-b632-4f59-bf0d-682cc6122188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:39.101425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41ceb-b632-4f59-bf0d-682cc6122188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:39.102840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41ceb-b632-4f59-bf0d-682cc6122188 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ff8a6f58-7149-47d8-a674-abe9ae430455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:39.107026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:39.108287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:39.109378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:39.109378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:39.109378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:39.112336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 291/590] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:39.113840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a6f58-7149-47d8-a674-abe9ae430455 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:39.116942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 300/591] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:39.117831 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f41ceb-b632-4f59-bf0d-682cc6122188 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:39.141968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:39.229437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f41ceb-b632-4f59-bf0d-682cc6122188 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:39.230120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a6f58-7149-47d8-a674-abe9ae430455 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:39.266029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784db9d-6204-49d8-b2e7-a10d6c4ae5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:39.266636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784db9d-6204-49d8-b2e7-a10d6c4ae5fa None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.266923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784db9d-6204-49d8-b2e7-a10d6c4ae5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.267199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784db9d-6204-49d8-b2e7-a10d6c4ae5fa None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.267902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784db9d-6204-49d8-b2e7-a10d6c4ae5fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:39.268255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784db9d-6204-49d8-b2e7-a10d6c4ae5fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:39.269690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784db9d-6204-49d8-b2e7-a10d6c4ae5fa None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:39.271393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1939f5-3f9c-46fb-85e9-d26a1d206c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:39.272256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:39.272682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1939f5-3f9c-46fb-85e9-d26a1d206c7f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.272965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1939f5-3f9c-46fb-85e9-d26a1d206c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.273239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1939f5-3f9c-46fb-85e9-d26a1d206c7f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.283580 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 301/592] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:39.284336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784db9d-6204-49d8-b2e7-a10d6c4ae5fa None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:39.284708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1939f5-3f9c-46fb-85e9-d26a1d206c7f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:39.286717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1939f5-3f9c-46fb-85e9-d26a1d206c7f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7d1939f5-3f9c-46fb-85e9-d26a1d206c7f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:39.299975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:39.301165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:39.302116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:39.302116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:39.302116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:39.302116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:39.302116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:39.302116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:39.302116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:39.302116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:39.302116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:50:39.302116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:39.302116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 292/593] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:39.302711 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1939f5-3f9c-46fb-85e9-d26a1d206c7f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:39.383780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:39.400271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784db9d-6204-49d8-b2e7-a10d6c4ae5fa None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:39.417424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1939f5-3f9c-46fb-85e9-d26a1d206c7f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:39.440183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891d4c1-5693-4296-8b01-54914f5baf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:39.440794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891d4c1-5693-4296-8b01-54914f5baf4b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.441082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891d4c1-5693-4296-8b01-54914f5baf4b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.441352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891d4c1-5693-4296-8b01-54914f5baf4b None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.441992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891d4c1-5693-4296-8b01-54914f5baf4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:39.442302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891d4c1-5693-4296-8b01-54914f5baf4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:39.443676 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891d4c1-5693-4296-8b01-54914f5baf4b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:39.454743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f062a-86fd-4161-aa84-2b0cddc2963e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:39.455457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f062a-86fd-4161-aa84-2b0cddc2963e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.455740 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f062a-86fd-4161-aa84-2b0cddc2963e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.456014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f062a-86fd-4161-aa84-2b0cddc2963e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.460940 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 302/594] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:39.461763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891d4c1-5693-4296-8b01-54914f5baf4b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:39.464292 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f062a-86fd-4161-aa84-2b0cddc2963e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:39.465749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f062a-86fd-4161-aa84-2b0cddc2963e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-9b8f062a-86fd-4161-aa84-2b0cddc2963e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:39.477198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:39.478140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:39.479652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:39.479652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:39.479652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:39.479652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:39.479652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:39.479652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:39.479652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:39.479652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:39.479652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 293/595] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:39.479652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f062a-86fd-4161-aa84-2b0cddc2963e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:39.530687 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:39.578024 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891d4c1-5693-4296-8b01-54914f5baf4b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:39.589255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f062a-86fd-4161-aa84-2b0cddc2963e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:39.614378 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e258c4-766c-489f-9c76-d28ea6e22ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:39.615027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e258c4-766c-489f-9c76-d28ea6e22ed2 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.615302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e258c4-766c-489f-9c76-d28ea6e22ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.615571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e258c4-766c-489f-9c76-d28ea6e22ed2 None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/794dbf38682e4b4f939854eecbacaf07/roles/080424a21d984f0eb115d19bbd53af20` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.623554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e428f0-23eb-4f21-a58c-bdda08f6ff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:39.624224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e428f0-23eb-4f21-a58c-bdda08f6ff79 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.624512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e428f0-23eb-4f21-a58c-bdda08f6ff79 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.624789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e428f0-23eb-4f21-a58c-bdda08f6ff79 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.625672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e428f0-23eb-4f21-a58c-bdda08f6ff79 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:39.625979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e428f0-23eb-4f21-a58c-bdda08f6ff79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:39.627410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e428f0-23eb-4f21-a58c-bdda08f6ff79 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:39.629701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e258c4-766c-489f-9c76-d28ea6e22ed2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=794dbf38682e4b4f939854eecbacaf07, role_id=080424a21d984f0eb115d19bbd53af20)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:39.630088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:39.631243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e258c4-766c-489f-9c76-d28ea6e22ed2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:39.644614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 303/596] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:50:39 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/794dbf38682e4b4f939854eecbacaf07/roles/080424a21d984f0eb115d19bbd53af20 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:50:39.645416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e258c4-766c-489f-9c76-d28ea6e22ed2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:39.651639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 294/597] 10.222.0.86 () {56 vars in 1139 bytes} [Wed Aug 24 09:50:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:39.651639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e428f0-23eb-4f21-a58c-bdda08f6ff79 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:39.761418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e258c4-766c-489f-9c76-d28ea6e22ed2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:39.770534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:39.773798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e428f0-23eb-4f21-a58c-bdda08f6ff79 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:39.812084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390d8bc-0504-4890-b989-fba9050f6ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:39.812715 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390d8bc-0504-4890-b989-fba9050f6ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.812993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390d8bc-0504-4890-b989-fba9050f6ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.813259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390d8bc-0504-4890-b989-fba9050f6ce8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.818765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508f52b-dd28-4c41-b164-58e45306a182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:39.819675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f52b-dd28-4c41-b164-58e45306a182 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.820028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f52b-dd28-4c41-b164-58e45306a182 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.820411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f52b-dd28-4c41-b164-58e45306a182 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.820973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390d8bc-0504-4890-b989-fba9050f6ce8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:39.821140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508f52b-dd28-4c41-b164-58e45306a182 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:39.821436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508f52b-dd28-4c41-b164-58e45306a182 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:39.822990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390d8bc-0504-4890-b989-fba9050f6ce8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:39.823650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508f52b-dd28-4c41-b164-58e45306a182 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c390d8bc-0504-4890-b989-fba9050f6ce8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:39.834325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:39.835252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:39.836574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:39.836574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:39.836574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:39.836574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:39.836574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:39.836574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:39.836574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:39.836574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:39.836574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 304/598] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:39.837337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d504f-fdf3-4f44-a3e6-ef69a4030f5d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.837337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d504f-fdf3-4f44-a3e6-ef69a4030f5d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.837631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d504f-fdf3-4f44-a3e6-ef69a4030f5d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.838969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 305/599] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:39.839467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d504f-fdf3-4f44-a3e6-ef69a4030f5d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:39.840358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 295/600] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:39.845138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288c815-8db7-4c04-8342-0ef7f697b1e3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:39.845427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288c815-8db7-4c04-8342-0ef7f697b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:39.845699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288c815-8db7-4c04-8342-0ef7f697b1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:39.865526 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9288c815-8db7-4c04-8342-0ef7f697b1e3 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:39.866732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9288c815-8db7-4c04-8342-0ef7f697b1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:39.920700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:39.941923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 296/601] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:39.942803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288c815-8db7-4c04-8342-0ef7f697b1e3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:39.972182 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d504f-fdf3-4f44-a3e6-ef69a4030f5d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:40.026715 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19563636-083e-4b7f-a709-16b4c12d43cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:40.027564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19563636-083e-4b7f-a709-16b4c12d43cf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:40.028488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19563636-083e-4b7f-a709-16b4c12d43cf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:40.028914 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19563636-083e-4b7f-a709-16b4c12d43cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:40.043807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19563636-083e-4b7f-a709-16b4c12d43cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:40.046106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19563636-083e-4b7f-a709-16b4c12d43cf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:40.053054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288c815-8db7-4c04-8342-0ef7f697b1e3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-19563636-083e-4b7f-a709-16b4c12d43cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:40.067856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:40.069057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:40.070184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:40.070184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:40.070184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:40.072728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 306/602] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:40.073691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19563636-083e-4b7f-a709-16b4c12d43cf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:40.100016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a033d4-9e73-4d53-bf70-d47e05a8bf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:40.100793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a033d4-9e73-4d53-bf70-d47e05a8bf6d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:40.100979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:40.101289 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a033d4-9e73-4d53-bf70-d47e05a8bf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:40.101843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a033d4-9e73-4d53-bf70-d47e05a8bf6d None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:40.102445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a033d4-9e73-4d53-bf70-d47e05a8bf6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:40.103644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a033d4-9e73-4d53-bf70-d47e05a8bf6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:40.105334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a033d4-9e73-4d53-bf70-d47e05a8bf6d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:40.122455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 297/603] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:40.124111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a033d4-9e73-4d53-bf70-d47e05a8bf6d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:40.186216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:40.194788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19563636-083e-4b7f-a709-16b4c12d43cf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:40.233011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45618ae6-eaeb-4f2e-b29a-96be438d9e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:40.233658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45618ae6-eaeb-4f2e-b29a-96be438d9e88 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:40.233934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45618ae6-eaeb-4f2e-b29a-96be438d9e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:40.234205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45618ae6-eaeb-4f2e-b29a-96be438d9e88 None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/98e11312493643b199fca04cb2d77853/roles/080424a21d984f0eb115d19bbd53af20` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:40.249598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45618ae6-eaeb-4f2e-b29a-96be438d9e88 None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=98e11312493643b199fca04cb2d77853, role_id=080424a21d984f0eb115d19bbd53af20)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:40.251085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45618ae6-eaeb-4f2e-b29a-96be438d9e88 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:40.254696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a033d4-9e73-4d53-bf70-d47e05a8bf6d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:40.276085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 307/604] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:50:40 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/98e11312493643b199fca04cb2d77853/roles/080424a21d984f0eb115d19bbd53af20 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:50:40.276085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45618ae6-eaeb-4f2e-b29a-96be438d9e88 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:40.292324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e3ab7-5460-4de4-9f52-3785a5e92aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:40.292907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e3ab7-5460-4de4-9f52-3785a5e92aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:40.293181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e3ab7-5460-4de4-9f52-3785a5e92aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:40.293630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e3ab7-5460-4de4-9f52-3785a5e92aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:40.294146 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e3ab7-5460-4de4-9f52-3785a5e92aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:40.294448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e3ab7-5460-4de4-9f52-3785a5e92aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:40.295878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e3ab7-5460-4de4-9f52-3785a5e92aa4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:40.309881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 298/605] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:40.310743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e3ab7-5460-4de4-9f52-3785a5e92aa4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:40.382987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:40.436483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45618ae6-eaeb-4f2e-b29a-96be438d9e88 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:40.436682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e3ab7-5460-4de4-9f52-3785a5e92aa4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:40.439823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:40.483815 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aa1c8-f755-48e1-a9a9-07c7fd32547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:40.484571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa1c8-f755-48e1-a9a9-07c7fd32547f None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:40.484852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa1c8-f755-48e1-a9a9-07c7fd32547f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:40.485155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa1c8-f755-48e1-a9a9-07c7fd32547f None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/bd7b580705f247699aeb58772e668730/roles/080424a21d984f0eb115d19bbd53af20` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:40.486262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4164d-57a7-4dfc-9173-16f3909d1f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:40.486893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4164d-57a7-4dfc-9173-16f3909d1f75 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:40.487168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4164d-57a7-4dfc-9173-16f3909d1f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:40.487434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4164d-57a7-4dfc-9173-16f3909d1f75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:40.501103 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4164d-57a7-4dfc-9173-16f3909d1f75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:40.502748 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512aa1c8-f755-48e1-a9a9-07c7fd32547f None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=bd7b580705f247699aeb58772e668730, role_id=080424a21d984f0eb115d19bbd53af20)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:40.504238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512aa1c8-f755-48e1-a9a9-07c7fd32547f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e3c4164d-57a7-4dfc-9173-16f3909d1f75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:40.505894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:40.506834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:40.507682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:40.508348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 299/606] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:40.509036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4164d-57a7-4dfc-9173-16f3909d1f75 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:40.522856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 308/607] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:50:40 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/bd7b580705f247699aeb58772e668730/roles/080424a21d984f0eb115d19bbd53af20 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:50:40.524014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aa1c8-f755-48e1-a9a9-07c7fd32547f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:40.662253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4164d-57a7-4dfc-9173-16f3909d1f75 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:40.706937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727aa1f-3d24-4573-b0ea-a5c992b275a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:40.707587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727aa1f-3d24-4573-b0ea-a5c992b275a8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:40.707873 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727aa1f-3d24-4573-b0ea-a5c992b275a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:40.708193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727aa1f-3d24-4573-b0ea-a5c992b275a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:40.716963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727aa1f-3d24-4573-b0ea-a5c992b275a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:40.718541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727aa1f-3d24-4573-b0ea-a5c992b275a8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:40.729604 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aa1c8-f755-48e1-a9a9-07c7fd32547f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-1727aa1f-3d24-4573-b0ea-a5c992b275a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:40.734686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:40.735651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:40.736868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:40.736868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:40.736868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:40.736868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:40.736868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:40.736868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:40.736868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:40.736868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:40.736868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 300/608] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:40.750453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727aa1f-3d24-4573-b0ea-a5c992b275a8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:40.760266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:40.795240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57680551-dbb1-4166-bbd5-1b23b27e087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:40.795857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57680551-dbb1-4166-bbd5-1b23b27e087c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:40.796199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57680551-dbb1-4166-bbd5-1b23b27e087c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:40.796575 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57680551-dbb1-4166-bbd5-1b23b27e087c None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:40.797220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57680551-dbb1-4166-bbd5-1b23b27e087c None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:40.797524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57680551-dbb1-4166-bbd5-1b23b27e087c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:40.798137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:40.800496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57680551-dbb1-4166-bbd5-1b23b27e087c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:40.822593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 309/609] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:40.841769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57680551-dbb1-4166-bbd5-1b23b27e087c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:40.895628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727aa1f-3d24-4573-b0ea-a5c992b275a8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:40.940834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f975597-e85b-40eb-8d42-69742ca230dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:40.943227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f975597-e85b-40eb-8d42-69742ca230dc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:40.944614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f975597-e85b-40eb-8d42-69742ca230dc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:40.945059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f975597-e85b-40eb-8d42-69742ca230dc None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:40.947191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f975597-e85b-40eb-8d42-69742ca230dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:40.948653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f975597-e85b-40eb-8d42-69742ca230dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:40.950780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f975597-e85b-40eb-8d42-69742ca230dc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:40.979648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 301/610] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:40.985176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f975597-e85b-40eb-8d42-69742ca230dc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:41.019892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57680551-dbb1-4166-bbd5-1b23b27e087c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:41.060835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d37e4c-c22d-42d0-a3fb-4896514cef45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:41.061482 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d37e4c-c22d-42d0-a3fb-4896514cef45 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:41.061739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d37e4c-c22d-42d0-a3fb-4896514cef45 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:41.062006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d37e4c-c22d-42d0-a3fb-4896514cef45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:41.089916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d37e4c-c22d-42d0-a3fb-4896514cef45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:41.091381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d37e4c-c22d-42d0-a3fb-4896514cef45 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:41.120453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-f0d37e4c-c22d-42d0-a3fb-4896514cef45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:41.133380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:41.135112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:41.136005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:41.136005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:41.136005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:41.136005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 310/611] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:41.144708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f975597-e85b-40eb-8d42-69742ca230dc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:41.153063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d37e4c-c22d-42d0-a3fb-4896514cef45 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:41.184554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c2965-ab28-4061-bee4-48a4e3f53e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:41.185311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c2965-ab28-4061-bee4-48a4e3f53e0b None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:41.185666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c2965-ab28-4061-bee4-48a4e3f53e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:41.186263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c2965-ab28-4061-bee4-48a4e3f53e0b None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/27cc091f3b094cb1a1e2650a3fbd7671/roles/080424a21d984f0eb115d19bbd53af20` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:41.203826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c2965-ab28-4061-bee4-48a4e3f53e0b None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=27cc091f3b094cb1a1e2650a3fbd7671, role_id=080424a21d984f0eb115d19bbd53af20)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:41.205503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c2965-ab28-4061-bee4-48a4e3f53e0b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:41.225321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 302/612] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:50:40 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/27cc091f3b094cb1a1e2650a3fbd7671/roles/080424a21d984f0eb115d19bbd53af20 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:50:41.226874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:41.314574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d37e4c-c22d-42d0-a3fb-4896514cef45 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:41.367790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64aa25-bfd2-420b-b33a-6b2ca00e1450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:41.368890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64aa25-bfd2-420b-b33a-6b2ca00e1450 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:41.369381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64aa25-bfd2-420b-b33a-6b2ca00e1450 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:41.369858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64aa25-bfd2-420b-b33a-6b2ca00e1450 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:41.370794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64aa25-bfd2-420b-b33a-6b2ca00e1450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:41.371313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64aa25-bfd2-420b-b33a-6b2ca00e1450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:41.373155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64aa25-bfd2-420b-b33a-6b2ca00e1450 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:41.398806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 311/613] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:41.405292 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c2965-ab28-4061-bee4-48a4e3f53e0b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:41.533167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c2965-ab28-4061-bee4-48a4e3f53e0b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:41.568635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fb90e-722e-44f2-9399-f349329a4cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:41.569261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb90e-722e-44f2-9399-f349329a4cbc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:41.569536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb90e-722e-44f2-9399-f349329a4cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:41.569805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb90e-722e-44f2-9399-f349329a4cbc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:41.572367 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:41.582177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fb90e-722e-44f2-9399-f349329a4cbc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:41.583733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fb90e-722e-44f2-9399-f349329a4cbc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a89fb90e-722e-44f2-9399-f349329a4cbc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:41.598057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:41.599242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:41.600197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:41.600197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:41.600197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:41.600197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:41.600197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:41.600197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:41.600197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:41.600197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:41.600197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:50:41.600197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:41.603527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 303/614] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:41.609136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64aa25-bfd2-420b-b33a-6b2ca00e1450 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:41.769287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:41.781088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64aa25-bfd2-420b-b33a-6b2ca00e1450 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:41.830704 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf1922-e296-4405-a76f-893db34f5270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:41.831319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf1922-e296-4405-a76f-893db34f5270 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:41.831546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf1922-e296-4405-a76f-893db34f5270 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:41.831768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf1922-e296-4405-a76f-893db34f5270 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:41.832532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf1922-e296-4405-a76f-893db34f5270 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:41.832842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf1922-e296-4405-a76f-893db34f5270 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:41.834288 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf1922-e296-4405-a76f-893db34f5270 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:41.852433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 312/615] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Aug 24 09:50:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:41.858623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fb90e-722e-44f2-9399-f349329a4cbc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:41.896155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47144bf2-67cd-48cd-87a6-3044bb9dea90 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:41.896479 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47144bf2-67cd-48cd-87a6-3044bb9dea90 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:41.896717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47144bf2-67cd-48cd-87a6-3044bb9dea90 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:41.898282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 313/616] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:41.903480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe311f9-8b3d-4518-bebc-bd9386055a1c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:41.903770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe311f9-8b3d-4518-bebc-bd9386055a1c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:41.904118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe311f9-8b3d-4518-bebc-bd9386055a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:41.922640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cbe311f9-8b3d-4518-bebc-bd9386055a1c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:41.924141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cbe311f9-8b3d-4518-bebc-bd9386055a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:42.014834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fb90e-722e-44f2-9399-f349329a4cbc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:42.054444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 314/617] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:42.079528 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67be4bf-6b30-42c6-b72d-204a09c5838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:42.080197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67be4bf-6b30-42c6-b72d-204a09c5838a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:42.080483 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67be4bf-6b30-42c6-b72d-204a09c5838a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:42.080811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67be4bf-6b30-42c6-b72d-204a09c5838a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:42.097993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67be4bf-6b30-42c6-b72d-204a09c5838a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:42.099468 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67be4bf-6b30-42c6-b72d-204a09c5838a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c67be4bf-6b30-42c6-b72d-204a09c5838a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:42.120640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:42.121604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:42.122549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:42.122549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:42.122549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:42.122549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 304/618] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:42.127480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe311f9-8b3d-4518-bebc-bd9386055a1c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:42.211631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:42.243700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe311f9-8b3d-4518-bebc-bd9386055a1c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:42.295437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687462c-3814-454f-8857-8021ef9ee145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:42.296469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687462c-3814-454f-8857-8021ef9ee145 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:42.296967 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687462c-3814-454f-8857-8021ef9ee145 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:42.297431 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687462c-3814-454f-8857-8021ef9ee145 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:42.298287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687462c-3814-454f-8857-8021ef9ee145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:42.298793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687462c-3814-454f-8857-8021ef9ee145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:42.300638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687462c-3814-454f-8857-8021ef9ee145 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:42.318610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 315/619] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:42.324637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67be4bf-6b30-42c6-b72d-204a09c5838a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:42.357000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687462c-3814-454f-8857-8021ef9ee145 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:42.408682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:42.467408 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67be4bf-6b30-42c6-b72d-204a09c5838a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:42.500303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687462c-3814-454f-8857-8021ef9ee145 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:42.502417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e48278-dde4-418e-b516-138672e1c282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:42.503040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48278-dde4-418e-b516-138672e1c282 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:42.503315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48278-dde4-418e-b516-138672e1c282 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:42.503584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48278-dde4-418e-b516-138672e1c282 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:42.519150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e48278-dde4-418e-b516-138672e1c282 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:42.528760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e48278-dde4-418e-b516-138672e1c282 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:42.535273 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d410910-2cb6-4dce-b6f5-0663e0b86289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:42.535902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d410910-2cb6-4dce-b6f5-0663e0b86289 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:42.536214 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d410910-2cb6-4dce-b6f5-0663e0b86289 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:42.536488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d410910-2cb6-4dce-b6f5-0663e0b86289 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:42.547215 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d410910-2cb6-4dce-b6f5-0663e0b86289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-98e48278-dde4-418e-b516-138672e1c282 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:42.548604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:42.549910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:42.550777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:42.550777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:42.550777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:42.550777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:42.550777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:42.550777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:42.550777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:42.550777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:42.550777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 305/620] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-4d410910-2cb6-4dce-b6f5-0663e0b86289 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:42.552668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:42.554688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 316/621] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:42.560315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:42.562084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e48278-dde4-418e-b516-138672e1c282 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:42.562229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d410910-2cb6-4dce-b6f5-0663e0b86289 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:42.662495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d410910-2cb6-4dce-b6f5-0663e0b86289 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:42.697718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe0b11-a6cd-4f86-804e-e530ffa51bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:42.698330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe0b11-a6cd-4f86-804e-e530ffa51bdb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:42.698607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe0b11-a6cd-4f86-804e-e530ffa51bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:42.698879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe0b11-a6cd-4f86-804e-e530ffa51bdb None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:42.699549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe0b11-a6cd-4f86-804e-e530ffa51bdb None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:42.699853 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe0b11-a6cd-4f86-804e-e530ffa51bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:42.701263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe0b11-a6cd-4f86-804e-e530ffa51bdb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:42.709947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 317/622] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:42.715834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe0b11-a6cd-4f86-804e-e530ffa51bdb None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:42.750342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e48278-dde4-418e-b516-138672e1c282 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:42.760960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:42.782231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:42.801614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef8493-5328-4092-9608-b7be59bcb9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:42.802611 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef8493-5328-4092-9608-b7be59bcb9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:42.803633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef8493-5328-4092-9608-b7be59bcb9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:42.804144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef8493-5328-4092-9608-b7be59bcb9e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:42.805481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef8493-5328-4092-9608-b7be59bcb9e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:42.805991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef8493-5328-4092-9608-b7be59bcb9e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:42.808939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef8493-5328-4092-9608-b7be59bcb9e8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:42.824988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 306/623] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:42.826493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4e781-6691-4a93-baf6-0f26513c9ef6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:42.827013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4e781-6691-4a93-baf6-0f26513c9ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:42.827471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4e781-6691-4a93-baf6-0f26513c9ef6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:42.831875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe0b11-a6cd-4f86-804e-e530ffa51bdb None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:42.833266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 307/624] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:42.834692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310af713-f9ec-4a8d-8094-7a6dc7add598 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:42.834979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310af713-f9ec-4a8d-8094-7a6dc7add598 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:42.835256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310af713-f9ec-4a8d-8094-7a6dc7add598 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:42.836877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 308/625] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:42.837605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310af713-f9ec-4a8d-8094-7a6dc7add598 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:42.871016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b1d09-40bd-468b-93a8-f3f9d7762a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:42.871901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b1d09-40bd-468b-93a8-f3f9d7762a29 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:42.872208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b1d09-40bd-468b-93a8-f3f9d7762a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:42.872485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b1d09-40bd-468b-93a8-f3f9d7762a29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:42.883040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b1d09-40bd-468b-93a8-f3f9d7762a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:42.885078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b1d09-40bd-468b-93a8-f3f9d7762a29 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-876b1d09-40bd-468b-93a8-f3f9d7762a29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:42.899294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:42.900374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:42.901281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:42.901281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:42.901281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:42.901281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 318/626] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:42.902013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f742134-3652-4ec5-9f24-0ee14693e494 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:42.902013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f742134-3652-4ec5-9f24-0ee14693e494 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:42.902137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f742134-3652-4ec5-9f24-0ee14693e494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:42.920454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7f742134-3652-4ec5-9f24-0ee14693e494 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:42.921572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7f742134-3652-4ec5-9f24-0ee14693e494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:42.955333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310af713-f9ec-4a8d-8094-7a6dc7add598 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:42.991419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4ce4d-0021-44ef-8490-a5829a66f27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:42.992089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4ce4d-0021-44ef-8490-a5829a66f27c None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:42.992369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4ce4d-0021-44ef-8490-a5829a66f27c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:42.992649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4ce4d-0021-44ef-8490-a5829a66f27c None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/f1b1d85195db46b19d6a0156dbdf7f6e/roles/080424a21d984f0eb115d19bbd53af20` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:42.998856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 319/627] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:43.000384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ce54f-d2e2-4b9a-a040-9c9f7b0fadbd None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:43.000835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ce54f-d2e2-4b9a-a040-9c9f7b0fadbd None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:43.001261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ce54f-d2e2-4b9a-a040-9c9f7b0fadbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:43.017559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4ce4d-0021-44ef-8490-a5829a66f27c None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=f1b1d85195db46b19d6a0156dbdf7f6e, role_id=080424a21d984f0eb115d19bbd53af20)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:43.019701 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4ce4d-0021-44ef-8490-a5829a66f27c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:43.022768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f19ce54f-d2e2-4b9a-a040-9c9f7b0fadbd None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:43.024552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f19ce54f-d2e2-4b9a-a040-9c9f7b0fadbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:43.044490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 309/628] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:50:42 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/f1b1d85195db46b19d6a0156dbdf7f6e/roles/080424a21d984f0eb115d19bbd53af20 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:50:43.045775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4ce4d-0021-44ef-8490-a5829a66f27c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:43.084658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:43.087938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:43.140425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 320/629] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:43.237399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4ce4d-0021-44ef-8490-a5829a66f27c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:43.274463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424a7ad-648d-452b-8302-a40ee17f71a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:43.275104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424a7ad-648d-452b-8302-a40ee17f71a8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:43.275383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424a7ad-648d-452b-8302-a40ee17f71a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:43.275655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424a7ad-648d-452b-8302-a40ee17f71a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:43.276319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424a7ad-648d-452b-8302-a40ee17f71a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:43.276622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424a7ad-648d-452b-8302-a40ee17f71a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:43.277987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424a7ad-648d-452b-8302-a40ee17f71a8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:43.294048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 310/630] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:43.302369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ce54f-d2e2-4b9a-a040-9c9f7b0fadbd None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:43.329564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424a7ad-648d-452b-8302-a40ee17f71a8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:43.446162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:43.455891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ce54f-d2e2-4b9a-a040-9c9f7b0fadbd None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:43.494567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424a7ad-648d-452b-8302-a40ee17f71a8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:43.503729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064f84f-e83b-453f-a939-9075a1c6cafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:43.504716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064f84f-e83b-453f-a939-9075a1c6cafc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:43.505592 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064f84f-e83b-453f-a939-9075a1c6cafc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:43.506064 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064f84f-e83b-453f-a939-9075a1c6cafc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:43.516473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064f84f-e83b-453f-a939-9075a1c6cafc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:43.518790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064f84f-e83b-453f-a939-9075a1c6cafc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:43.533735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7eae1-aab8-4b69-90fb-a5157acc4945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:43.534368 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7eae1-aab8-4b69-90fb-a5157acc4945 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-d064f84f-e83b-453f-a939-9075a1c6cafc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:43.534632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:43.535540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:43.536534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:43.536534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:43.536534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:43.536534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:43.536534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:43.536534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:43.536534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:43.536534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:43.536534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:50:43.536534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:43.537068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7eae1-aab8-4b69-90fb-a5157acc4945 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:43.537068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7eae1-aab8-4b69-90fb-a5157acc4945 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:43.537749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 321/631] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:43.538043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064f84f-e83b-453f-a939-9075a1c6cafc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:43.550753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7eae1-aab8-4b69-90fb-a5157acc4945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-77c7eae1-aab8-4b69-90fb-a5157acc4945 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:43.558927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:43.559874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 311/632] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:43.561183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7eae1-aab8-4b69-90fb-a5157acc4945 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:43.585770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:43.657495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064f84f-e83b-453f-a939-9075a1c6cafc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:43.670697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7eae1-aab8-4b69-90fb-a5157acc4945 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:43.693867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7b3f4-33c0-4d91-af4a-aefa10be97e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:43.694612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7b3f4-33c0-4d91-af4a-aefa10be97e8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:43.694943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7b3f4-33c0-4d91-af4a-aefa10be97e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:43.695233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7b3f4-33c0-4d91-af4a-aefa10be97e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:43.707160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7b3f4-33c0-4d91-af4a-aefa10be97e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:43.709682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9b640-ec89-4be1-8a11-5988e0118b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:43.710291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b640-ec89-4be1-8a11-5988e0118b8d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:43.710572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b640-ec89-4be1-8a11-5988e0118b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:43.710849 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b640-ec89-4be1-8a11-5988e0118b8d None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:43.711592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9b640-ec89-4be1-8a11-5988e0118b8d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:43.711902 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9b640-ec89-4be1-8a11-5988e0118b8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:43.713414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9b640-ec89-4be1-8a11-5988e0118b8d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-21d7b3f4-33c0-4d91-af4a-aefa10be97e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:43.714564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:43.715454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:43.716969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:43.717598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 322/633] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:43.723722 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7b3f4-33c0-4d91-af4a-aefa10be97e8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:43.724291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:43.745917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 312/634] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Aug 24 09:50:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:43.747109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9b640-ec89-4be1-8a11-5988e0118b8d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:43.838934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:43.846427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7b3f4-33c0-4d91-af4a-aefa10be97e8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:43.863364 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9b640-ec89-4be1-8a11-5988e0118b8d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:43.884156 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd332d-9693-4c08-b2f2-feef744a588d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:43.885730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd332d-9693-4c08-b2f2-feef744a588d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:43.886018 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd332d-9693-4c08-b2f2-feef744a588d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:43.886446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd332d-9693-4c08-b2f2-feef744a588d None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:43.887116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd332d-9693-4c08-b2f2-feef744a588d None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:43.887444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd332d-9693-4c08-b2f2-feef744a588d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:43.888855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd332d-9693-4c08-b2f2-feef744a588d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:43.902115 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 323/635] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:43.903466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c2e4-d712-4e6f-9667-0cba59d73e90 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:43.903755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c2e4-d712-4e6f-9667-0cba59d73e90 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:43.903934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049c1d9-2fa8-4d18-8eae-dc1f163df781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:43.904188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c2e4-d712-4e6f-9667-0cba59d73e90 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:43.904587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049c1d9-2fa8-4d18-8eae-dc1f163df781 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:43.904862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049c1d9-2fa8-4d18-8eae-dc1f163df781 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:43.905132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049c1d9-2fa8-4d18-8eae-dc1f163df781 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:43.908248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 324/636] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:43.908384 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049c1d9-2fa8-4d18-8eae-dc1f163df781 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:43.908384 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049c1d9-2fa8-4d18-8eae-dc1f163df781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:43.908521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9c2e4-d712-4e6f-9667-0cba59d73e90 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:43.908591 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049c1d9-2fa8-4d18-8eae-dc1f163df781 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:43.917542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 313/637] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:43.918366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049c1d9-2fa8-4d18-8eae-dc1f163df781 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:43.982353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:44.023870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9c2e4-d712-4e6f-9667-0cba59d73e90 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:44.032677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049c1d9-2fa8-4d18-8eae-dc1f163df781 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:44.061050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b42efa-2c95-4879-b27a-106c5f4ad920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:44.061798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b42efa-2c95-4879-b27a-106c5f4ad920 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:44.062078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b42efa-2c95-4879-b27a-106c5f4ad920 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:44.062351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b42efa-2c95-4879-b27a-106c5f4ad920 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:44.068448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6b27c-2cad-49e2-912a-df7722b1ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:44.069076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b27c-2cad-49e2-912a-df7722b1ed71 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:44.069361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b27c-2cad-49e2-912a-df7722b1ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:44.069637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b27c-2cad-49e2-912a-df7722b1ed71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:44.080545 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6b27c-2cad-49e2-912a-df7722b1ed71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:44.081988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6b27c-2cad-49e2-912a-df7722b1ed71 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:44.088303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-afd6b27c-2cad-49e2-912a-df7722b1ed71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:44.097344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:44.098460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:44.099350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:44.099350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:44.099350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:44.099350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 314/638] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:44.099595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e427bd-a26d-4a2b-9915-3bddca6941f4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:44.100299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e427bd-a26d-4a2b-9915-3bddca6941f4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:44.100299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e427bd-a26d-4a2b-9915-3bddca6941f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:44.118761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a9e427bd-a26d-4a2b-9915-3bddca6941f4 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:44.120783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a9e427bd-a26d-4a2b-9915-3bddca6941f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:44.142154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b42efa-2c95-4879-b27a-106c5f4ad920 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:44.143844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b42efa-2c95-4879-b27a-106c5f4ad920 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:44.193614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 315/639] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:44.194428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e427bd-a26d-4a2b-9915-3bddca6941f4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:44.262102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:44.278768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 325/640] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:44.279593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b42efa-2c95-4879-b27a-106c5f4ad920 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:44.340357 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e427bd-a26d-4a2b-9915-3bddca6941f4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:44.345386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:44.397283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5ed6a-dd74-4b25-b3f2-05c904438140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:44.398194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5ed6a-dd74-4b25-b3f2-05c904438140 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:44.398476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5ed6a-dd74-4b25-b3f2-05c904438140 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:44.398755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5ed6a-dd74-4b25-b3f2-05c904438140 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:44.406977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b42efa-2c95-4879-b27a-106c5f4ad920 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:44.413066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5ed6a-dd74-4b25-b3f2-05c904438140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:44.414541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5ed6a-dd74-4b25-b3f2-05c904438140 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7df5ed6a-dd74-4b25-b3f2-05c904438140 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:44.434819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:44.435833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:44.437210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:44.437210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:44.437210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:44.437210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 316/641] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:44.437993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5ed6a-dd74-4b25-b3f2-05c904438140 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:44.457856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f48a0-7a3e-4753-bd1f-b70af74c2337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:44.458466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f48a0-7a3e-4753-bd1f-b70af74c2337 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:44.458750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f48a0-7a3e-4753-bd1f-b70af74c2337 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:44.459028 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f48a0-7a3e-4753-bd1f-b70af74c2337 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:44.459936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f48a0-7a3e-4753-bd1f-b70af74c2337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:44.460410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f48a0-7a3e-4753-bd1f-b70af74c2337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:44.462183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f48a0-7a3e-4753-bd1f-b70af74c2337 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:44.480495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 326/642] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:44.481356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f48a0-7a3e-4753-bd1f-b70af74c2337 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:44.539361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:44.550055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5ed6a-dd74-4b25-b3f2-05c904438140 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:44.585775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc341f28-dfa9-48f4-9485-dfaebf9fabaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:44.586394 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc341f28-dfa9-48f4-9485-dfaebf9fabaf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:44.586670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc341f28-dfa9-48f4-9485-dfaebf9fabaf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:44.586939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc341f28-dfa9-48f4-9485-dfaebf9fabaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:44.598134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc341f28-dfa9-48f4-9485-dfaebf9fabaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:44.599763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc341f28-dfa9-48f4-9485-dfaebf9fabaf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:44.602579 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f48a0-7a3e-4753-bd1f-b70af74c2337 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bc341f28-dfa9-48f4-9485-dfaebf9fabaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:44.614678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:44.615746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:44.616924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:44.616924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:44.616924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:44.616924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 317/643] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:44.616924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc341f28-dfa9-48f4-9485-dfaebf9fabaf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:44.626494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:44.646481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b09378-2d19-4736-8546-0ab8edaf50b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:44.647085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09378-2d19-4736-8546-0ab8edaf50b7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:44.647365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09378-2d19-4736-8546-0ab8edaf50b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:44.647884 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b09378-2d19-4736-8546-0ab8edaf50b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:44.648305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b09378-2d19-4736-8546-0ab8edaf50b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:44.648606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b09378-2d19-4736-8546-0ab8edaf50b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:44.649970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b09378-2d19-4736-8546-0ab8edaf50b7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:44.664140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 327/644] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:44.664873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b09378-2d19-4736-8546-0ab8edaf50b7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:44.738505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc341f28-dfa9-48f4-9485-dfaebf9fabaf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:44.778968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60412e03-0495-480b-874d-8730c9b609a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:44.779603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60412e03-0495-480b-874d-8730c9b609a1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:44.779907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60412e03-0495-480b-874d-8730c9b609a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:44.780187 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60412e03-0495-480b-874d-8730c9b609a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:44.788209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60412e03-0495-480b-874d-8730c9b609a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:44.789733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60412e03-0495-480b-874d-8730c9b609a1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:44.799985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b09378-2d19-4736-8546-0ab8edaf50b7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-60412e03-0495-480b-874d-8730c9b609a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:44.801068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:44.802203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:44.803529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:44.803529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:44.803529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:44.803529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:44.803529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:44.803529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:44.803529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:44.803529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:44.803529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:50:44.803529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:44.806966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 318/645] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Aug 24 09:50:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:44.807825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60412e03-0495-480b-874d-8730c9b609a1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:44.810682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:44.842528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cc688-f46c-4b8e-8d71-e60a445092d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:44.843155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cc688-f46c-4b8e-8d71-e60a445092d0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:44.843416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cc688-f46c-4b8e-8d71-e60a445092d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:44.843687 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cc688-f46c-4b8e-8d71-e60a445092d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:44.854595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cc688-f46c-4b8e-8d71-e60a445092d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-0d8cc688-f46c-4b8e-8d71-e60a445092d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:44.859410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:44.860672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:44.861754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:44.863449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 328/646] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:44.864198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cc688-f46c-4b8e-8d71-e60a445092d0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:44.903220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:44.927039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60412e03-0495-480b-874d-8730c9b609a1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:44.966378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1bcac-9332-4ce6-8f87-dfaabfa7e317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:44.967088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1bcac-9332-4ce6-8f87-dfaabfa7e317 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:44.967437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1bcac-9332-4ce6-8f87-dfaabfa7e317 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:44.967787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1bcac-9332-4ce6-8f87-dfaabfa7e317 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:44.968533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1bcac-9332-4ce6-8f87-dfaabfa7e317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:44.968920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1bcac-9332-4ce6-8f87-dfaabfa7e317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:44.970309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1bcac-9332-4ce6-8f87-dfaabfa7e317 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:44.975843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cc688-f46c-4b8e-8d71-e60a445092d0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:44.984062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 319/647] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:44.984919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1bcac-9332-4ce6-8f87-dfaabfa7e317 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:45.018148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3d808-2393-4d87-a88e-cae04cba4a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:45.018774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3d808-2393-4d87-a88e-cae04cba4a67 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.019114 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3d808-2393-4d87-a88e-cae04cba4a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.019391 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3d808-2393-4d87-a88e-cae04cba4a67 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.027515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3d808-2393-4d87-a88e-cae04cba4a67 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:45.029025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3d808-2393-4d87-a88e-cae04cba4a67 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-f9e3d808-2393-4d87-a88e-cae04cba4a67 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:45.041425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:45.042547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:45.043768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:45.043768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:45.043768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:45.043768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:45.043768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:45.043768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:45.043768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:45.043768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:45.043768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:50:45.043768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:45.047200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 329/648] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:45.048120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3d808-2393-4d87-a88e-cae04cba4a67 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:45.063312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:45.105743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1bcac-9332-4ce6-8f87-dfaabfa7e317 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:45.140510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cd950-b92b-48c3-b263-ca9582d7a75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:45.141263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cd950-b92b-48c3-b263-ca9582d7a75b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.141622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cd950-b92b-48c3-b263-ca9582d7a75b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.141963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cd950-b92b-48c3-b263-ca9582d7a75b None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.142764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cd950-b92b-48c3-b263-ca9582d7a75b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:45.143145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cd950-b92b-48c3-b263-ca9582d7a75b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:45.144797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cd950-b92b-48c3-b263-ca9582d7a75b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:45.155111 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:45.167866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 320/649] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Aug 24 09:50:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:45.168073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3d808-2393-4d87-a88e-cae04cba4a67 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:45.168201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cd950-b92b-48c3-b263-ca9582d7a75b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:45.204106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfe2c1-dc33-45e6-893f-bdb6fa56fc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:45.204708 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfe2c1-dc33-45e6-893f-bdb6fa56fc5f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.204989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfe2c1-dc33-45e6-893f-bdb6fa56fc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.205262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfe2c1-dc33-45e6-893f-bdb6fa56fc5f None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.205961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfe2c1-dc33-45e6-893f-bdb6fa56fc5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:45.206232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfe2c1-dc33-45e6-893f-bdb6fa56fc5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:45.207630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfe2c1-dc33-45e6-893f-bdb6fa56fc5f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:45.220807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 330/650] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:45.220807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfe2c1-dc33-45e6-893f-bdb6fa56fc5f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:45.270147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cd950-b92b-48c3-b263-ca9582d7a75b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:45.290407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:45.307334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41f1b2-d499-4eb1-9a68-0e8f1c85943b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:45.307881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41f1b2-d499-4eb1-9a68-0e8f1c85943b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.308186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41f1b2-d499-4eb1-9a68-0e8f1c85943b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.308465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41f1b2-d499-4eb1-9a68-0e8f1c85943b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.314958 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41f1b2-d499-4eb1-9a68-0e8f1c85943b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:45.316414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41f1b2-d499-4eb1-9a68-0e8f1c85943b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:45.317130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfe2c1-dc33-45e6-893f-bdb6fa56fc5f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6d41f1b2-d499-4eb1-9a68-0e8f1c85943b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:45.327115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:45.328221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:45.329137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:45.329137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:45.329137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:45.329137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:45.329137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:45.329137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:45.329137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:45.329137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:45.329137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 321/651] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:45.329634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41f1b2-d499-4eb1-9a68-0e8f1c85943b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:45.354498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55c47b-e184-438d-a7a0-0bf210b9f9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:45.355089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55c47b-e184-438d-a7a0-0bf210b9f9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.355364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55c47b-e184-438d-a7a0-0bf210b9f9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.355634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55c47b-e184-438d-a7a0-0bf210b9f9b0 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.356380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55c47b-e184-438d-a7a0-0bf210b9f9b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:45.356686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55c47b-e184-438d-a7a0-0bf210b9f9b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:45.358105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55c47b-e184-438d-a7a0-0bf210b9f9b0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:45.374670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 331/652] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Aug 24 09:50:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:45.380672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b915-144c-437e-afc2-93932b403cf3 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.380961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b915-144c-437e-afc2-93932b403cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.381235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b915-144c-437e-afc2-93932b403cf3 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.382486 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 332/653] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:45.383187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263b915-144c-437e-afc2-93932b403cf3 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:45.401507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:45.428382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41f1b2-d499-4eb1-9a68-0e8f1c85943b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:45.462196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f51a9c-6a9b-4581-a77c-efd2da422e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:45.462804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f51a9c-6a9b-4581-a77c-efd2da422e40 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.463258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f51a9c-6a9b-4581-a77c-efd2da422e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.463428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f51a9c-6a9b-4581-a77c-efd2da422e40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.484837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263b915-144c-437e-afc2-93932b403cf3 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:45.512469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:45.519957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d4419-6010-47e0-a2ee-616a065a578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:45.520606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d4419-6010-47e0-a2ee-616a065a578a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.520885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d4419-6010-47e0-a2ee-616a065a578a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.521158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d4419-6010-47e0-a2ee-616a065a578a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.531156 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d4419-6010-47e0-a2ee-616a065a578a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:45.532038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f51a9c-6a9b-4581-a77c-efd2da422e40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:45.532659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d4419-6010-47e0-a2ee-616a065a578a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:45.534062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f51a9c-6a9b-4581-a77c-efd2da422e40 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-6a0d4419-6010-47e0-a2ee-616a065a578a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:45.547854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:45.549027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:45.550249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:45.550249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:45.550249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:45.554149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 333/654] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:45.554761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d4419-6010-47e0-a2ee-616a065a578a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:45.620449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:45.635273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 322/655] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:45.636143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f51a9c-6a9b-4581-a77c-efd2da422e40 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:45.658315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d4419-6010-47e0-a2ee-616a065a578a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:45.693768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8603b51-3eef-4933-ae78-abe2b086b7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:45.694479 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8603b51-3eef-4933-ae78-abe2b086b7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.694644 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8603b51-3eef-4933-ae78-abe2b086b7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.694917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8603b51-3eef-4933-ae78-abe2b086b7a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.695608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8603b51-3eef-4933-ae78-abe2b086b7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:45.695921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8603b51-3eef-4933-ae78-abe2b086b7a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:45.697416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8603b51-3eef-4933-ae78-abe2b086b7a3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:45.709078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 334/656] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:45.710236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be738-97eb-4b70-b390-394e1daef4e9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.710519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be738-97eb-4b70-b390-394e1daef4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.710795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be738-97eb-4b70-b390-394e1daef4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.718213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:45.727792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f09be738-97eb-4b70-b390-394e1daef4e9 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:45.729551 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f09be738-97eb-4b70-b390-394e1daef4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:45.738533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f51a9c-6a9b-4581-a77c-efd2da422e40 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:45.773778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03398045-dbc7-47ae-abc7-c179c62322f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:45.774379 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03398045-dbc7-47ae-abc7-c179c62322f0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.774658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03398045-dbc7-47ae-abc7-c179c62322f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.774929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03398045-dbc7-47ae-abc7-c179c62322f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.798459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 335/657] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:45.798459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09be738-97eb-4b70-b390-394e1daef4e9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:45.846489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03398045-dbc7-47ae-abc7-c179c62322f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:45.848237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03398045-dbc7-47ae-abc7-c179c62322f0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:45.868347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:45.917607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09be738-97eb-4b70-b390-394e1daef4e9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:45.944981 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:45.954369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f399a-e2b5-4f9a-b8f2-be4f9a8d2a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:45.954980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f399a-e2b5-4f9a-b8f2-be4f9a8d2a7f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:45.955260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f399a-e2b5-4f9a-b8f2-be4f9a8d2a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:45.955533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f399a-e2b5-4f9a-b8f2-be4f9a8d2a7f None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:45.956213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f399a-e2b5-4f9a-b8f2-be4f9a8d2a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:45.956521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f399a-e2b5-4f9a-b8f2-be4f9a8d2a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:45.957891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f399a-e2b5-4f9a-b8f2-be4f9a8d2a7f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:45.967290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 323/658] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:45.968063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03398045-dbc7-47ae-abc7-c179c62322f0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:45.977482 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 336/659] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:45.979625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f399a-e2b5-4f9a-b8f2-be4f9a8d2a7f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:46.079074 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03398045-dbc7-47ae-abc7-c179c62322f0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:46.082522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f399a-e2b5-4f9a-b8f2-be4f9a8d2a7f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:46.099500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:46.114092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28866f-1f2e-4f64-a96c-298ca413b161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:46.114738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28866f-1f2e-4f64-a96c-298ca413b161 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:46.115016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28866f-1f2e-4f64-a96c-298ca413b161 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:46.115293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28866f-1f2e-4f64-a96c-298ca413b161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:46.117390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69a8e7-6266-41a6-9dbb-c95ddb4a4834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:46.117990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69a8e7-6266-41a6-9dbb-c95ddb4a4834 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:46.118265 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69a8e7-6266-41a6-9dbb-c95ddb4a4834 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:46.118533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69a8e7-6266-41a6-9dbb-c95ddb4a4834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:46.119267 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69a8e7-6266-41a6-9dbb-c95ddb4a4834 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=794dbf38682e4b4f939854eecbacaf07) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:46.119577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69a8e7-6266-41a6-9dbb-c95ddb4a4834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:46.121051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69a8e7-6266-41a6-9dbb-c95ddb4a4834 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:46.125484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28866f-1f2e-4f64-a96c-298ca413b161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:46.126895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28866f-1f2e-4f64-a96c-298ca413b161 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:46.140101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 337/660] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:45 2022] GET /identity/v3/role_assignments?user.id=794dbf38682e4b4f939854eecbacaf07&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:46.140101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69a8e7-6266-41a6-9dbb-c95ddb4a4834 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0f28866f-1f2e-4f64-a96c-298ca413b161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:46.155544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:46.156612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:46.157491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:46.157491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:46.157491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:46.157491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 324/661] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:46.157726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28866f-1f2e-4f64-a96c-298ca413b161 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:46.200410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:46.261117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69a8e7-6266-41a6-9dbb-c95ddb4a4834 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:46.281004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28866f-1f2e-4f64-a96c-298ca413b161 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:46.302587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b6e19-4772-418e-8b85-0ddc82430e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:46.303233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b6e19-4772-418e-8b85-0ddc82430e8f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:46.303516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b6e19-4772-418e-8b85-0ddc82430e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:46.303790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b6e19-4772-418e-8b85-0ddc82430e8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:46.318066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b6e19-4772-418e-8b85-0ddc82430e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:46.319541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b6e19-4772-418e-8b85-0ddc82430e8f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:46.321403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1145c4-4846-4cec-bdfe-7cd3738d01ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:46.322098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1145c4-4846-4cec-bdfe-7cd3738d01ab None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:46.322389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1145c4-4846-4cec-bdfe-7cd3738d01ab None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:46.323201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1145c4-4846-4cec-bdfe-7cd3738d01ab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:46.330978 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1145c4-4846-4cec-bdfe-7cd3738d01ab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:46.332467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1145c4-4846-4cec-bdfe-7cd3738d01ab None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c11b6e19-4772-418e-8b85-0ddc82430e8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:46.336343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:46.337468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:46.338982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:46.338982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:46.338982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:46.338982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 338/662] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:46.339217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b6e19-4772-418e-8b85-0ddc82430e8f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4f1145c4-4846-4cec-bdfe-7cd3738d01ab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:46.346936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:46.348241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:46.349296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:46.349296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:46.349296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:46.349296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:46.349296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:46.349296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:46.349296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:46.349296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:46.349296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:50:46.349296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:46.351540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 325/663] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:46.352873 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1145c4-4846-4cec-bdfe-7cd3738d01ab None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:46.378452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:46.453083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:46.463184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1145c4-4846-4cec-bdfe-7cd3738d01ab None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:46.464364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b6e19-4772-418e-8b85-0ddc82430e8f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:46.499082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7d606-cdf4-4905-9697-46a7bd180f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:46.499369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231fc7c-5932-43c1-ae55-a6697dba0edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:46.499700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7d606-cdf4-4905-9697-46a7bd180f68 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:46.499980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7d606-cdf4-4905-9697-46a7bd180f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:46.500276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231fc7c-5932-43c1-ae55-a6697dba0edc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:46.500381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7d606-cdf4-4905-9697-46a7bd180f68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:46.500725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231fc7c-5932-43c1-ae55-a6697dba0edc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:46.501114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231fc7c-5932-43c1-ae55-a6697dba0edc None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:46.501874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231fc7c-5932-43c1-ae55-a6697dba0edc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:46.502282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231fc7c-5932-43c1-ae55-a6697dba0edc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:46.503761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231fc7c-5932-43c1-ae55-a6697dba0edc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:46.511638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7d606-cdf4-4905-9697-46a7bd180f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-e8f7d606-cdf4-4905-9697-46a7bd180f68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:46.516878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:46.517980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:46.519007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:46.524391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 326/664] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:46.524391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231fc7c-5932-43c1-ae55-a6697dba0edc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:46.524617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 339/665] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:46.524617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7d606-cdf4-4905-9697-46a7bd180f68 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:46.624195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:46.632176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7d606-cdf4-4905-9697-46a7bd180f68 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:46.632342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231fc7c-5932-43c1-ae55-a6697dba0edc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:46.667900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988deb33-6cdb-467b-a0cf-d1f1ae034bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:46.668261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66adbab-9869-4a06-b5c2-c267c157e560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:46.668537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988deb33-6cdb-467b-a0cf-d1f1ae034bba None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:46.668719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66adbab-9869-4a06-b5c2-c267c157e560 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:46.668816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988deb33-6cdb-467b-a0cf-d1f1ae034bba None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:46.668994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66adbab-9869-4a06-b5c2-c267c157e560 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:46.669095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988deb33-6cdb-467b-a0cf-d1f1ae034bba None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:46.669269 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66adbab-9869-4a06-b5c2-c267c157e560 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:46.669796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988deb33-6cdb-467b-a0cf-d1f1ae034bba None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:46.669891 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66adbab-9869-4a06-b5c2-c267c157e560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:46.670103 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988deb33-6cdb-467b-a0cf-d1f1ae034bba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:46.670211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66adbab-9869-4a06-b5c2-c267c157e560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:46.671531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988deb33-6cdb-467b-a0cf-d1f1ae034bba None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:46.671629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66adbab-9869-4a06-b5c2-c267c157e560 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:46.688012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 327/666] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:46.688012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66adbab-9869-4a06-b5c2-c267c157e560 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:46.697258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 340/667] 10.222.0.86 () {56 vars in 1139 bytes} [Wed Aug 24 09:50:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:46.698521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988deb33-6cdb-467b-a0cf-d1f1ae034bba None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:46.703753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:46.804279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66adbab-9869-4a06-b5c2-c267c157e560 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:46.811126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988deb33-6cdb-467b-a0cf-d1f1ae034bba None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:46.841442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014337a2-4fcc-4181-bf99-7057a46d3838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:46.842041 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014337a2-4fcc-4181-bf99-7057a46d3838 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:46.842319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014337a2-4fcc-4181-bf99-7057a46d3838 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:46.842594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014337a2-4fcc-4181-bf99-7057a46d3838 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:46.843299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014337a2-4fcc-4181-bf99-7057a46d3838 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:46.843600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014337a2-4fcc-4181-bf99-7057a46d3838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:46.844987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014337a2-4fcc-4181-bf99-7057a46d3838 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:46.848328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4453c1c-bf0b-43ca-8885-c05fa0d988f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:46.848962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4453c1c-bf0b-43ca-8885-c05fa0d988f9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:46.849246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4453c1c-bf0b-43ca-8885-c05fa0d988f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:46.849516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4453c1c-bf0b-43ca-8885-c05fa0d988f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:46.857564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 328/668] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:46.857564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014337a2-4fcc-4181-bf99-7057a46d3838 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:46.857805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4453c1c-bf0b-43ca-8885-c05fa0d988f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:46.858451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4453c1c-bf0b-43ca-8885-c05fa0d988f9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-f4453c1c-bf0b-43ca-8885-c05fa0d988f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:46.870202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:46.871873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:46.872848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:46.872848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:46.872848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:46.872848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:46.872848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:46.872848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:46.872848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:46.872848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:46.872848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 341/669] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:46.872848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4453c1c-bf0b-43ca-8885-c05fa0d988f9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:46.889934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:46.942843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:46.971143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014337a2-4fcc-4181-bf99-7057a46d3838 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:46.981771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4453c1c-bf0b-43ca-8885-c05fa0d988f9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:47.009604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79e0cf-943d-4ecd-9177-94917af0a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:47.010248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79e0cf-943d-4ecd-9177-94917af0a58b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:47.010637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79e0cf-943d-4ecd-9177-94917af0a58b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:47.010909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79e0cf-943d-4ecd-9177-94917af0a58b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:47.016262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df17af9-526c-420b-bfcb-c3c70f7c2368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:47.016860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df17af9-526c-420b-bfcb-c3c70f7c2368 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:47.017144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df17af9-526c-420b-bfcb-c3c70f7c2368 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:47.017416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df17af9-526c-420b-bfcb-c3c70f7c2368 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:47.017706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79e0cf-943d-4ecd-9177-94917af0a58b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:47.019266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79e0cf-943d-4ecd-9177-94917af0a58b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-df79e0cf-943d-4ecd-9177-94917af0a58b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:47.031352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:47.032281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:47.033189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:47.033189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:47.033189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:47.033189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:47.033189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:47.033189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:47.033189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:47.033189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:47.033189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 329/670] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:47.034407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79e0cf-943d-4ecd-9177-94917af0a58b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:47.093728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df17af9-526c-420b-bfcb-c3c70f7c2368 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:47.095194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df17af9-526c-420b-bfcb-c3c70f7c2368 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:47.110465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:47.144253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79e0cf-943d-4ecd-9177-94917af0a58b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:47.181835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8074c-4bfb-4049-bb3e-0b4eafd780be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:47.183011 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8074c-4bfb-4049-bb3e-0b4eafd780be None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:47.183286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8074c-4bfb-4049-bb3e-0b4eafd780be None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:47.183771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8074c-4bfb-4049-bb3e-0b4eafd780be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:47.186337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:47.195398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8074c-4bfb-4049-bb3e-0b4eafd780be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:47.196880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8074c-4bfb-4049-bb3e-0b4eafd780be None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:47.220071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 342/671] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5bc8074c-4bfb-4049-bb3e-0b4eafd780be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:47.220298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:47.221480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:47.222597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:47.222597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:47.222597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:47.222819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df17af9-526c-420b-bfcb-c3c70f7c2368 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:47.222910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 330/672] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:47.222910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8074c-4bfb-4049-bb3e-0b4eafd780be None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:47.327524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df17af9-526c-420b-bfcb-c3c70f7c2368 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:47.334742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8074c-4bfb-4049-bb3e-0b4eafd780be None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:47.342244 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:47.365636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6759e2-b3fb-4b9e-bcfb-546fc7ee8702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:47.366229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6759e2-b3fb-4b9e-bcfb-546fc7ee8702 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:47.366502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6759e2-b3fb-4b9e-bcfb-546fc7ee8702 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:47.366775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6759e2-b3fb-4b9e-bcfb-546fc7ee8702 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:47.367450 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6759e2-b3fb-4b9e-bcfb-546fc7ee8702 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:47.367747 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6759e2-b3fb-4b9e-bcfb-546fc7ee8702 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:47.369195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6759e2-b3fb-4b9e-bcfb-546fc7ee8702 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:47.370822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4823f6-4c10-4649-a412-7bd5a7df78c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:47.371410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4823f6-4c10-4649-a412-7bd5a7df78c1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:47.371689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4823f6-4c10-4649-a412-7bd5a7df78c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:47.371957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4823f6-4c10-4649-a412-7bd5a7df78c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:47.372665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4823f6-4c10-4649-a412-7bd5a7df78c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:47.372963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4823f6-4c10-4649-a412-7bd5a7df78c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:47.374367 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4823f6-4c10-4649-a412-7bd5a7df78c1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:47.385884 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 343/673] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:47.385884 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6759e2-b3fb-4b9e-bcfb-546fc7ee8702 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:47.391938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 331/674] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:47.396953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4823f6-4c10-4649-a412-7bd5a7df78c1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:47.446140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:47.504396 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6759e2-b3fb-4b9e-bcfb-546fc7ee8702 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:47.505299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4823f6-4c10-4649-a412-7bd5a7df78c1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:47.546311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2e612-a474-46f7-927f-c20bcb8ce452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:47.547061 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2e612-a474-46f7-927f-c20bcb8ce452 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:47.547260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73601abd-cf15-480e-9cfc-b07662b3db35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:47.547390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2e612-a474-46f7-927f-c20bcb8ce452 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:47.547600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2e612-a474-46f7-927f-c20bcb8ce452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:47.547860 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73601abd-cf15-480e-9cfc-b07662b3db35 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:47.548171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73601abd-cf15-480e-9cfc-b07662b3db35 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:47.548445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73601abd-cf15-480e-9cfc-b07662b3db35 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:47.549093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73601abd-cf15-480e-9cfc-b07662b3db35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:47.549392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73601abd-cf15-480e-9cfc-b07662b3db35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:47.550999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73601abd-cf15-480e-9cfc-b07662b3db35 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:47.559147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2e612-a474-46f7-927f-c20bcb8ce452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:47.560670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2e612-a474-46f7-927f-c20bcb8ce452 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:47.567287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 332/675] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:47.567287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73601abd-cf15-480e-9cfc-b07662b3db35 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-17e2e612-a474-46f7-927f-c20bcb8ce452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:47.579614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:47.580601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:47.581523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:47.581523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:47.581523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:47.585729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 344/676] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:47.586375 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2e612-a474-46f7-927f-c20bcb8ce452 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:47.613498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:47.674303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73601abd-cf15-480e-9cfc-b07662b3db35 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:47.685202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:47.691845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2e612-a474-46f7-927f-c20bcb8ce452 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:47.716966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9758f7-515c-417e-ae78-83111c418fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:47.717564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9758f7-515c-417e-ae78-83111c418fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:47.717840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9758f7-515c-417e-ae78-83111c418fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:47.718109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9758f7-515c-417e-ae78-83111c418fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:47.718854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9758f7-515c-417e-ae78-83111c418fb1 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=98e11312493643b199fca04cb2d77853) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:47.719191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9758f7-515c-417e-ae78-83111c418fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:47.720650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9758f7-515c-417e-ae78-83111c418fb1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:47.727106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 333/677] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:47 2022] GET /identity/v3/role_assignments?user.id=98e11312493643b199fca04cb2d77853&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:47.727902 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9758f7-515c-417e-ae78-83111c418fb1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:47.739794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdbe45-c5fd-41b6-b97e-50b7274b91c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:47.740471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdbe45-c5fd-41b6-b97e-50b7274b91c6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:47.740751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdbe45-c5fd-41b6-b97e-50b7274b91c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:47.741025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdbe45-c5fd-41b6-b97e-50b7274b91c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:47.741783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdbe45-c5fd-41b6-b97e-50b7274b91c6 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=bd7b580705f247699aeb58772e668730) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:47.742096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdbe45-c5fd-41b6-b97e-50b7274b91c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:47.743770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdbe45-c5fd-41b6-b97e-50b7274b91c6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:47.756758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 345/678] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:47 2022] GET /identity/v3/role_assignments?user.id=bd7b580705f247699aeb58772e668730&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:47.757685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdbe45-c5fd-41b6-b97e-50b7274b91c6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:47.919762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9758f7-515c-417e-ae78-83111c418fb1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:47.964609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:47.970791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d89ee-7555-49c8-941d-63b790ab0445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:47.971567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d89ee-7555-49c8-941d-63b790ab0445 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:47.971730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d89ee-7555-49c8-941d-63b790ab0445 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:47.971972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d89ee-7555-49c8-941d-63b790ab0445 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:47.981197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d89ee-7555-49c8-941d-63b790ab0445 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:47.981494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdbe45-c5fd-41b6-b97e-50b7274b91c6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:47.982666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d89ee-7555-49c8-941d-63b790ab0445 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-885d89ee-7555-49c8-941d-63b790ab0445 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:47.995563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:47.996804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:47.997795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:47.997795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:47.997795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:47.997795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:47.997795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:47.997795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:47.997795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:47.997795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:47.997795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:50:47.997795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:47.997795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 334/679] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:48.002354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d89ee-7555-49c8-941d-63b790ab0445 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:48.025307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f740851-d0ef-442a-86f2-88802248b51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:48.026778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740851-d0ef-442a-86f2-88802248b51f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:48.027217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740851-d0ef-442a-86f2-88802248b51f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:48.027491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740851-d0ef-442a-86f2-88802248b51f None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:48.028836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f740851-d0ef-442a-86f2-88802248b51f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:48.028836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f740851-d0ef-442a-86f2-88802248b51f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:48.029861 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f740851-d0ef-442a-86f2-88802248b51f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:48.039856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:48.051227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 346/680] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:48.059520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f740851-d0ef-442a-86f2-88802248b51f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:48.123360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d89ee-7555-49c8-941d-63b790ab0445 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:48.167431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f605f-cf53-4dcb-b0cb-692fcaad187b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:48.168147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f605f-cf53-4dcb-b0cb-692fcaad187b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:48.168446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f605f-cf53-4dcb-b0cb-692fcaad187b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:48.168736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f605f-cf53-4dcb-b0cb-692fcaad187b None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:48.169487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f605f-cf53-4dcb-b0cb-692fcaad187b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:48.169794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f605f-cf53-4dcb-b0cb-692fcaad187b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:48.171245 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f605f-cf53-4dcb-b0cb-692fcaad187b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:48.192262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 335/681] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Aug 24 09:50:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:48.194279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439711e9-2b0f-4a07-8c1e-dc97130c54fb None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:48.194568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439711e9-2b0f-4a07-8c1e-dc97130c54fb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:48.194847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439711e9-2b0f-4a07-8c1e-dc97130c54fb None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:48.195166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f740851-d0ef-442a-86f2-88802248b51f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:48.196436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 336/682] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:48.200180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439711e9-2b0f-4a07-8c1e-dc97130c54fb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:48.242053 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61d2a-0209-4cbe-ba80-afd63bc59443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:48.242694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61d2a-0209-4cbe-ba80-afd63bc59443 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:48.243028 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61d2a-0209-4cbe-ba80-afd63bc59443 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:48.243368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61d2a-0209-4cbe-ba80-afd63bc59443 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:48.250986 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a61d2a-0209-4cbe-ba80-afd63bc59443 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:48.252671 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a61d2a-0209-4cbe-ba80-afd63bc59443 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-b1a61d2a-0209-4cbe-ba80-afd63bc59443 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:48.265330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:48.266272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:48.267177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:48.267177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:48.267177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:48.267177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:48.267177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:48.267177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:48.267177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:48.267177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:48.267177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 347/683] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:48.268149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bf599-f515-41a5-8f9f-f0bccad3ef1c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:48.268987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bf599-f515-41a5-8f9f-f0bccad3ef1c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:48.268987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bf599-f515-41a5-8f9f-f0bccad3ef1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:48.292251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-402bf599-f515-41a5-8f9f-f0bccad3ef1c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:48.293437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-402bf599-f515-41a5-8f9f-f0bccad3ef1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:48.295646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:48.309518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:48.324086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439711e9-2b0f-4a07-8c1e-dc97130c54fb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:48.367874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528233d7-60df-4c5f-aee8-95062d5eb950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:48.368515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528233d7-60df-4c5f-aee8-95062d5eb950 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:48.368790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528233d7-60df-4c5f-aee8-95062d5eb950 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:48.369061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528233d7-60df-4c5f-aee8-95062d5eb950 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:48.384416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 348/684] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:48.385406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bf599-f515-41a5-8f9f-f0bccad3ef1c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:48.445633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528233d7-60df-4c5f-aee8-95062d5eb950 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:48.447199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528233d7-60df-4c5f-aee8-95062d5eb950 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:48.541866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:48.557079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 337/685] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:50:48.562384 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528233d7-60df-4c5f-aee8-95062d5eb950 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:48.578145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bf599-f515-41a5-8f9f-f0bccad3ef1c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:48.661253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be4452-709c-4a43-8869-e4727545a2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:48.662251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be4452-709c-4a43-8869-e4727545a2fa None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:48.662733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be4452-709c-4a43-8869-e4727545a2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:48.663204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be4452-709c-4a43-8869-e4727545a2fa None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:48.664139 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be4452-709c-4a43-8869-e4727545a2fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:48.664709 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be4452-709c-4a43-8869-e4727545a2fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:48.666513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be4452-709c-4a43-8869-e4727545a2fa None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:48.677294 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528233d7-60df-4c5f-aee8-95062d5eb950 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:48.682564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:48.695666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 349/686] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:48.701260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be4452-709c-4a43-8869-e4727545a2fa None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:48.719635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8b4cf-a215-493e-baf6-1de91f07b03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:48.720511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b4cf-a215-493e-baf6-1de91f07b03b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:48.720916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b4cf-a215-493e-baf6-1de91f07b03b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:48.721483 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b4cf-a215-493e-baf6-1de91f07b03b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:48.733562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8b4cf-a215-493e-baf6-1de91f07b03b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:48.735470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8b4cf-a215-493e-baf6-1de91f07b03b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-71c8b4cf-a215-493e-baf6-1de91f07b03b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:48.757073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:48.758259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:48.759356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:48.759356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:48.759356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:48.767616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 338/687] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:48.767616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8b4cf-a215-493e-baf6-1de91f07b03b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:48.821030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:48.843894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be4452-709c-4a43-8869-e4727545a2fa None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:48.883907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738f351-458f-4bb8-8118-984c64f2e519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:48.892245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738f351-458f-4bb8-8118-984c64f2e519 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:48.892245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738f351-458f-4bb8-8118-984c64f2e519 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:48.892245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738f351-458f-4bb8-8118-984c64f2e519 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:48.892542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738f351-458f-4bb8-8118-984c64f2e519 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=27cc091f3b094cb1a1e2650a3fbd7671) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:48.893262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738f351-458f-4bb8-8118-984c64f2e519 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:48.894826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8b4cf-a215-493e-baf6-1de91f07b03b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:48.894968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738f351-458f-4bb8-8118-984c64f2e519 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:48.904141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 350/688] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:48 2022] GET /identity/v3/role_assignments?user.id=27cc091f3b094cb1a1e2650a3fbd7671&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:48.905357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738f351-458f-4bb8-8118-984c64f2e519 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:48.973064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21387fd6-b03c-44b3-9380-2f700248c242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:48.973713 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21387fd6-b03c-44b3-9380-2f700248c242 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:48.973991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21387fd6-b03c-44b3-9380-2f700248c242 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:48.974261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21387fd6-b03c-44b3-9380-2f700248c242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:48.998425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21387fd6-b03c-44b3-9380-2f700248c242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:48.999953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21387fd6-b03c-44b3-9380-2f700248c242 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-21387fd6-b03c-44b3-9380-2f700248c242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:49.025619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:49.026724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:49.027647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:49.027647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:49.027647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:49.038650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 339/689] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:49.039691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21387fd6-b03c-44b3-9380-2f700248c242 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:49.054392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:49.134389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738f351-458f-4bb8-8118-984c64f2e519 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:49.184714 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47a1b4-1a27-4cba-adb4-ea482427d040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:49.185817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47a1b4-1a27-4cba-adb4-ea482427d040 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:49.186097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47a1b4-1a27-4cba-adb4-ea482427d040 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:49.186642 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47a1b4-1a27-4cba-adb4-ea482427d040 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:49.187305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47a1b4-1a27-4cba-adb4-ea482427d040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:49.187606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47a1b4-1a27-4cba-adb4-ea482427d040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:49.189138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:49.189854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47a1b4-1a27-4cba-adb4-ea482427d040 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:49.204789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21387fd6-b03c-44b3-9380-2f700248c242 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:49.205565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 351/690] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:49.211188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47a1b4-1a27-4cba-adb4-ea482427d040 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:49.244504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325cbf9-51cb-4c41-8df1-57468d35defa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:49.245187 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325cbf9-51cb-4c41-8df1-57468d35defa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:49.245467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325cbf9-51cb-4c41-8df1-57468d35defa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:49.245741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325cbf9-51cb-4c41-8df1-57468d35defa None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:49.246397 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325cbf9-51cb-4c41-8df1-57468d35defa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:49.246701 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325cbf9-51cb-4c41-8df1-57468d35defa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:49.248311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325cbf9-51cb-4c41-8df1-57468d35defa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:49.262887 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 340/691] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:49.269423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325cbf9-51cb-4c41-8df1-57468d35defa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:49.450502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47a1b4-1a27-4cba-adb4-ea482427d040 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:49.462872 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325cbf9-51cb-4c41-8df1-57468d35defa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:49.497611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:49.519234 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd3032-667f-4f01-aeb1-7c3af2e66f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:49.519858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd3032-667f-4f01-aeb1-7c3af2e66f1b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:49.520246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd3032-667f-4f01-aeb1-7c3af2e66f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:49.520517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd3032-667f-4f01-aeb1-7c3af2e66f1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:49.528802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd3032-667f-4f01-aeb1-7c3af2e66f1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:49.529557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698c1a0-6779-4f7f-b347-fb99661dafd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:49.530164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698c1a0-6779-4f7f-b347-fb99661dafd7 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:49.530388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd3032-667f-4f01-aeb1-7c3af2e66f1b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:49.530557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698c1a0-6779-4f7f-b347-fb99661dafd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:49.530721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698c1a0-6779-4f7f-b347-fb99661dafd7 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:49.531326 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698c1a0-6779-4f7f-b347-fb99661dafd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:49.533251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698c1a0-6779-4f7f-b347-fb99661dafd7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-6ddd3032-667f-4f01-aeb1-7c3af2e66f1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:49.549851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:49.552232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:49.554312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:49.554312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:49.554312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:49.554312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:49.554312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:49.554312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:49.554312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:49.554312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:49.557181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 352/692] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:49.565139 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd3032-667f-4f01-aeb1-7c3af2e66f1b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:49.579355 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 341/693] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:50:49 2022] POST /identity/v3/users => generated 254 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:49.691842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd3032-667f-4f01-aeb1-7c3af2e66f1b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:49.730703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a52ff-803a-48ca-898f-f7dcdc980ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:49.731648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a52ff-803a-48ca-898f-f7dcdc980ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:49.732195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a52ff-803a-48ca-898f-f7dcdc980ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:49.732665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a52ff-803a-48ca-898f-f7dcdc980ff5 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:49.733563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a52ff-803a-48ca-898f-f7dcdc980ff5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:49.734138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a52ff-803a-48ca-898f-f7dcdc980ff5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:49.735876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a52ff-803a-48ca-898f-f7dcdc980ff5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:49.754349 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 353/694] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:49.760732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698c1a0-6779-4f7f-b347-fb99661dafd7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:49.781899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:49.886312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698c1a0-6779-4f7f-b347-fb99661dafd7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:49.922142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29785bad-1d45-4195-bb70-362f86805627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:49.922796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29785bad-1d45-4195-bb70-362f86805627 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:49.923081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29785bad-1d45-4195-bb70-362f86805627 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:49.923349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29785bad-1d45-4195-bb70-362f86805627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:49.924096 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29785bad-1d45-4195-bb70-362f86805627 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=f1b1d85195db46b19d6a0156dbdf7f6e) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:49.924408 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29785bad-1d45-4195-bb70-362f86805627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:49.926129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29785bad-1d45-4195-bb70-362f86805627 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:49.936229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 342/695] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:49 2022] GET /identity/v3/role_assignments?user.id=f1b1d85195db46b19d6a0156dbdf7f6e&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:50.091022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e846e-0fcf-4b61-b9c1-f26e242cfe56 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:50.092005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e846e-0fcf-4b61-b9c1-f26e242cfe56 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:50.092515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e846e-0fcf-4b61-b9c1-f26e242cfe56 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:50.094352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 354/696] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:50.099767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b4efe-8639-428c-a5e5-2c4d3f3b263a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:50.100070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b4efe-8639-428c-a5e5-2c4d3f3b263a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:50.100348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b4efe-8639-428c-a5e5-2c4d3f3b263a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:50.125097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ab6b4efe-8639-428c-a5e5-2c4d3f3b263a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:50.126293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab6b4efe-8639-428c-a5e5-2c4d3f3b263a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:50.227987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 343/697] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:50.353383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d878c-dc36-4d3d-8258-bcb4a3c2e1c2 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:50.353655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d878c-dc36-4d3d-8258-bcb4a3c2e1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:50.353935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d878c-dc36-4d3d-8258-bcb4a3c2e1c2 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:50.356198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 355/698] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:50.362272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17af6-1d41-4116-8db3-89124fd465ca None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:50.362561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17af6-1d41-4116-8db3-89124fd465ca None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:50.362831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17af6-1d41-4116-8db3-89124fd465ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:50.369449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:50.382389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-55f17af6-1d41-4116-8db3-89124fd465ca None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:50.383598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-55f17af6-1d41-4116-8db3-89124fd465ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:50.457282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 344/699] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:50.563819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d878c-dc36-4d3d-8258-bcb4a3c2e1c2 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:50.622801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:50.705014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d878c-dc36-4d3d-8258-bcb4a3c2e1c2 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:50.748460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26c017-e6c8-408c-902f-5097c6acab5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:50.749096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26c017-e6c8-408c-902f-5097c6acab5c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:50.749379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26c017-e6c8-408c-902f-5097c6acab5c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:50.749649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26c017-e6c8-408c-902f-5097c6acab5c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:50.750241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26c017-e6c8-408c-902f-5097c6acab5c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:50.751616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26c017-e6c8-408c-902f-5097c6acab5c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:50.756827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f17af6-1d41-4116-8db3-89124fd465ca None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-cf26c017-e6c8-408c-902f-5097c6acab5c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:50.767549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:50.768628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:50:50.769528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:50.772846 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 356/700] 10.222.0.86 () {56 vars in 1129 bytes} [Wed Aug 24 09:50:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:50.778615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26c017-e6c8-408c-902f-5097c6acab5c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:50.889923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26c017-e6c8-408c-902f-5097c6acab5c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:50.898496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:50.948368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1519eee-5bde-43c0-8427-465967cf8774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:50.950559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1519eee-5bde-43c0-8427-465967cf8774 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:50.951034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1519eee-5bde-43c0-8427-465967cf8774 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:50.951764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1519eee-5bde-43c0-8427-465967cf8774 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:50.953310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1519eee-5bde-43c0-8427-465967cf8774 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:50.955074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1519eee-5bde-43c0-8427-465967cf8774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:50.959008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1519eee-5bde-43c0-8427-465967cf8774 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:50.967843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 357/701] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Aug 24 09:50:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:50.970195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f17af6-1d41-4116-8db3-89124fd465ca None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:50.974744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1519eee-5bde-43c0-8427-465967cf8774 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:51.031657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:51.041841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cffa49-a494-4ce6-9b98-8bf856553903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:51.048810 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cffa49-a494-4ce6-9b98-8bf856553903 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:51.049093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cffa49-a494-4ce6-9b98-8bf856553903 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:51.049370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cffa49-a494-4ce6-9b98-8bf856553903 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:51.081037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cffa49-a494-4ce6-9b98-8bf856553903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-f7cffa49-a494-4ce6-9b98-8bf856553903 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:51.086460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:51.087391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:51.092905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 345/702] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:51.099752 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cffa49-a494-4ce6-9b98-8bf856553903 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:51.124887 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1519eee-5bde-43c0-8427-465967cf8774 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:51.164749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad711d68-242f-4faf-8e89-469738da0cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:51.165975 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad711d68-242f-4faf-8e89-469738da0cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:51.166453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad711d68-242f-4faf-8e89-469738da0cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:51.167560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad711d68-242f-4faf-8e89-469738da0cc6 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:51.168573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad711d68-242f-4faf-8e89-469738da0cc6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:51.169474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad711d68-242f-4faf-8e89-469738da0cc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:51.171267 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad711d68-242f-4faf-8e89-469738da0cc6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:51.179080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 358/703] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Aug 24 09:50:50 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:51.263537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cffa49-a494-4ce6-9b98-8bf856553903 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:51.304245 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d709067-392d-464e-a4d8-1b7219c9b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:51.305233 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d709067-392d-464e-a4d8-1b7219c9b428 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:51.305710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d709067-392d-464e-a4d8-1b7219c9b428 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:51.306172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d709067-392d-464e-a4d8-1b7219c9b428 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:51.307049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d709067-392d-464e-a4d8-1b7219c9b428 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:51.307672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d709067-392d-464e-a4d8-1b7219c9b428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:51.309282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d709067-392d-464e-a4d8-1b7219c9b428 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:51.320036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 346/704] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:51.326065 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad711d68-242f-4faf-8e89-469738da0cc6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:51.341998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae012daf-b91e-4424-b39c-56d0cc967c04 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:51.342344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae012daf-b91e-4424-b39c-56d0cc967c04 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:51.342562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae012daf-b91e-4424-b39c-56d0cc967c04 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:51.347723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 347/705] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:51.350603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc6168-8e18-484b-81ac-7c230119768e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:51.351048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc6168-8e18-484b-81ac-7c230119768e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:51.353178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc6168-8e18-484b-81ac-7c230119768e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:51.392295 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8cbc6168-8e18-484b-81ac-7c230119768e None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:51.397042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8cbc6168-8e18-484b-81ac-7c230119768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:51.402255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:51.523381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad711d68-242f-4faf-8e89-469738da0cc6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:51.528398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:51.537641 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 348/706] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:51.602688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295753-a4aa-4c1f-a3a5-b584cd0e95ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:51.603640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295753-a4aa-4c1f-a3a5-b584cd0e95ea None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:51.604084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295753-a4aa-4c1f-a3a5-b584cd0e95ea None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:51.604604 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295753-a4aa-4c1f-a3a5-b584cd0e95ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:51.620709 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8295753-a4aa-4c1f-a3a5-b584cd0e95ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:51.622385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8295753-a4aa-4c1f-a3a5-b584cd0e95ea None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-f8295753-a4aa-4c1f-a3a5-b584cd0e95ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:51.665446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:51.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:51.667937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:51.667937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:51.667937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:51.670245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 359/707] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:51.677156 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc6168-8e18-484b-81ac-7c230119768e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:51.899452 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc6168-8e18-484b-81ac-7c230119768e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:51.913775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295753-a4aa-4c1f-a3a5-b584cd0e95ea None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:52.008755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbac227-3be0-4d31-a771-ae003969d939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:52.009755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbac227-3be0-4d31-a771-ae003969d939 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:52.010232 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbac227-3be0-4d31-a771-ae003969d939 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:52.010704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbac227-3be0-4d31-a771-ae003969d939 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:52.012213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbac227-3be0-4d31-a771-ae003969d939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:52.012728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbac227-3be0-4d31-a771-ae003969d939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:52.014303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbac227-3be0-4d31-a771-ae003969d939 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:52.017236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:52.036383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295753-a4aa-4c1f-a3a5-b584cd0e95ea None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:52.041262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 349/708] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:52.046711 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbac227-3be0-4d31-a771-ae003969d939 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:52.078371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69c736-9d83-46bf-95ea-104b95e9e064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:52.079265 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69c736-9d83-46bf-95ea-104b95e9e064 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:52.080131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69c736-9d83-46bf-95ea-104b95e9e064 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:52.080592 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69c736-9d83-46bf-95ea-104b95e9e064 None admin] PATH_INFO: `/v3/services/network` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:52.081732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69c736-9d83-46bf-95ea-104b95e9e064 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:52.083410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69c736-9d83-46bf-95ea-104b95e9e064 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ee69c736-9d83-46bf-95ea-104b95e9e064 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:52.093851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:52.095143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:50:52.096213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:52.099049 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 360/709] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:51 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:52.106177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69c736-9d83-46bf-95ea-104b95e9e064 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:52.160685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:52.226594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69c736-9d83-46bf-95ea-104b95e9e064 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:52.266823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7da37-b503-4801-b388-daccf24988dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:52.267973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7da37-b503-4801-b388-daccf24988dd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:52.268631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7da37-b503-4801-b388-daccf24988dd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:52.269258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7da37-b503-4801-b388-daccf24988dd None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:52.270246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7da37-b503-4801-b388-daccf24988dd None admin] RBAC: Adding query filter params (name=network) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:52.270743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7da37-b503-4801-b388-daccf24988dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:52.272343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7da37-b503-4801-b388-daccf24988dd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:52.278993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 361/710] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:50:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:52.287028 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7da37-b503-4801-b388-daccf24988dd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:52.298673 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbac227-3be0-4d31-a771-ae003969d939 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:52.309631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:52.336334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bfbf5-ce93-42c3-825e-a2b2e67d5c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:52.336569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bfbf5-ce93-42c3-825e-a2b2e67d5c9a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:52.336866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bfbf5-ce93-42c3-825e-a2b2e67d5c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:52.337076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bfbf5-ce93-42c3-825e-a2b2e67d5c9a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:52.344985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bfbf5-ce93-42c3-825e-a2b2e67d5c9a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:52.346588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bfbf5-ce93-42c3-825e-a2b2e67d5c9a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-104bfbf5-ce93-42c3-825e-a2b2e67d5c9a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:52.359552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:52.360535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:52.361488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:52.361488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:52.361488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:52.361488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:52.361488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:52.361488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:52.361488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:52.361488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:52.361488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:50:52.361488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:52.364377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 350/711] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Aug 24 09:50:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:52.365678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6cc7c-6915-44d9-b990-38119c40e746 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:52.366034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6cc7c-6915-44d9-b990-38119c40e746 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:52.366222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6cc7c-6915-44d9-b990-38119c40e746 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:52.369013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 351/712] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:52.369677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6cc7c-6915-44d9-b990-38119c40e746 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:52.425084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7da37-b503-4801-b388-daccf24988dd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:52.469288 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9032e-486d-4d3c-bb2d-fd4f3a670fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:52.470564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9032e-486d-4d3c-bb2d-fd4f3a670fdc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:52.471043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9032e-486d-4d3c-bb2d-fd4f3a670fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:52.471878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9032e-486d-4d3c-bb2d-fd4f3a670fdc None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:52.473127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9032e-486d-4d3c-bb2d-fd4f3a670fdc None admin] RBAC: Adding query filter params (type=network) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:52.473699 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9032e-486d-4d3c-bb2d-fd4f3a670fdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:52.476242 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9032e-486d-4d3c-bb2d-fd4f3a670fdc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:52.478823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6cc7c-6915-44d9-b990-38119c40e746 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:52.482515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:52.484432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 362/713] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:50:52 2022] GET /identity/v3/services?type=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:52.486188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c9cc6-749c-4a72-8c1f-82d732c25a1b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:52.486855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c9cc6-749c-4a72-8c1f-82d732c25a1b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:52.487309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c9cc6-749c-4a72-8c1f-82d732c25a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:52.510499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d41c9cc6-749c-4a72-8c1f-82d732c25a1b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:52.512098 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d41c9cc6-749c-4a72-8c1f-82d732c25a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:52.596824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 363/714] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:52.597923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b387607-f312-4c49-8d26-e03db23f77f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:52.598409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b387607-f312-4c49-8d26-e03db23f77f3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:52.598695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b387607-f312-4c49-8d26-e03db23f77f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:52.598970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b387607-f312-4c49-8d26-e03db23f77f3 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:52.599721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b387607-f312-4c49-8d26-e03db23f77f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:52.600032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b387607-f312-4c49-8d26-e03db23f77f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:52.601488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b387607-f312-4c49-8d26-e03db23f77f3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:52.643971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 352/715] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Aug 24 09:50:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:52.649517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c9cc6-749c-4a72-8c1f-82d732c25a1b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:52.671572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:52.730419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca33606-db00-4fc8-977a-59a874b211c4 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:52.730719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca33606-db00-4fc8-977a-59a874b211c4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:52.730979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca33606-db00-4fc8-977a-59a874b211c4 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:52.732661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 353/716] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:50:52.738494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0d452-014c-4bbe-bb95-70112e87d8f8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:52.739105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0d452-014c-4bbe-bb95-70112e87d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:52.739658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0d452-014c-4bbe-bb95-70112e87d8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:52.761438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-20f0d452-014c-4bbe-bb95-70112e87d8f8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:52.763043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-20f0d452-014c-4bbe-bb95-70112e87d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:52.779274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c9cc6-749c-4a72-8c1f-82d732c25a1b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:52.827542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a879f8e-7326-41c4-bc44-c816e294e365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:52.828170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879f8e-7326-41c4-bc44-c816e294e365 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:52.828444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879f8e-7326-41c4-bc44-c816e294e365 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:52.828716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879f8e-7326-41c4-bc44-c816e294e365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:52.846482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 354/717] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:50:52.922540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a879f8e-7326-41c4-bc44-c816e294e365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:52.924004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a879f8e-7326-41c4-bc44-c816e294e365 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:52.938185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:52.965043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0d452-014c-4bbe-bb95-70112e87d8f8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:53.061635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 364/718] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:53.067005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a879f8e-7326-41c4-bc44-c816e294e365 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:53.156948 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:53.173153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0d452-014c-4bbe-bb95-70112e87d8f8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:53.196494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a879f8e-7326-41c4-bc44-c816e294e365 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:53.210353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:53.246419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90258d-d0f6-418d-a675-56a558373c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:53.247461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90258d-d0f6-418d-a675-56a558373c14 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:53.247955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90258d-d0f6-418d-a675-56a558373c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:53.248461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90258d-d0f6-418d-a675-56a558373c14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:53.253398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43f53e-a686-4023-9eaa-995921ff3ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:53.254085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43f53e-a686-4023-9eaa-995921ff3ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:53.254366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43f53e-a686-4023-9eaa-995921ff3ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:53.254639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43f53e-a686-4023-9eaa-995921ff3ae2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:53.262351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90258d-d0f6-418d-a675-56a558373c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ae90258d-d0f6-418d-a675-56a558373c14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:53.267273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:53.268335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:53.269240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:53.269899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43f53e-a686-4023-9eaa-995921ff3ae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:53.269899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43f53e-a686-4023-9eaa-995921ff3ae2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:53.271933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 355/719] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:53.272924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90258d-d0f6-418d-a675-56a558373c14 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-5b43f53e-a686-4023-9eaa-995921ff3ae2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:53.287368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:53.288418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:53.289343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:53.289343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:53.289343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:53.289343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 365/720] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:53.289585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43f53e-a686-4023-9eaa-995921ff3ae2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:53.388999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90258d-d0f6-418d-a675-56a558373c14 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:53.426872 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9161e4-3a61-4c99-9f6a-271f1538a64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:53.427784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9161e4-3a61-4c99-9f6a-271f1538a64a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:53.428286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9161e4-3a61-4c99-9f6a-271f1538a64a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:53.428679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43f53e-a686-4023-9eaa-995921ff3ae2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:53.428937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9161e4-3a61-4c99-9f6a-271f1538a64a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:53.440278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9161e4-3a61-4c99-9f6a-271f1538a64a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3b9161e4-3a61-4c99-9f6a-271f1538a64a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:53.445229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:53.446154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:53.447005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:53.450003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 356/721] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:53.450958 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9161e4-3a61-4c99-9f6a-271f1538a64a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:53.467954 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:53.476363 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad66fc-ddd6-4191-8841-7285717184af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:53.477108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad66fc-ddd6-4191-8841-7285717184af None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:53.477459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad66fc-ddd6-4191-8841-7285717184af None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:53.477803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad66fc-ddd6-4191-8841-7285717184af None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:53.478628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad66fc-ddd6-4191-8841-7285717184af None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:53.479108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad66fc-ddd6-4191-8841-7285717184af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:53.480699 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad66fc-ddd6-4191-8841-7285717184af None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:53.493460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 366/722] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:53.494908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad66fc-ddd6-4191-8841-7285717184af None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:53.534412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:53.575049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9161e4-3a61-4c99-9f6a-271f1538a64a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:53.610716 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd3eb2-6911-4469-87d1-d3442a192aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:53.611375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd3eb2-6911-4469-87d1-d3442a192aeb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:53.611658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd3eb2-6911-4469-87d1-d3442a192aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:53.611935 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd3eb2-6911-4469-87d1-d3442a192aeb None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:53.612672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd3eb2-6911-4469-87d1-d3442a192aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:53.612980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd3eb2-6911-4469-87d1-d3442a192aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:53.614469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd3eb2-6911-4469-87d1-d3442a192aeb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:53.627163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad66fc-ddd6-4191-8841-7285717184af None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:53.632008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 357/723] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:53.633014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd3eb2-6911-4469-87d1-d3442a192aeb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:53.665269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91c30f-b1e0-4d13-ac9c-cd8faa1bd562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:53.665790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91c30f-b1e0-4d13-ac9c-cd8faa1bd562 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:53.666071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91c30f-b1e0-4d13-ac9c-cd8faa1bd562 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:53.666342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91c30f-b1e0-4d13-ac9c-cd8faa1bd562 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:53.667004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91c30f-b1e0-4d13-ac9c-cd8faa1bd562 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:53.667331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91c30f-b1e0-4d13-ac9c-cd8faa1bd562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:53.669395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91c30f-b1e0-4d13-ac9c-cd8faa1bd562 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:53.683269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 367/724] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:53.683269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91c30f-b1e0-4d13-ac9c-cd8faa1bd562 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:53.717401 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:53.748493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd3eb2-6911-4469-87d1-d3442a192aeb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:53.777879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:53.793807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26658fbd-23c5-4190-b965-70bcb7964dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:53.794922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26658fbd-23c5-4190-b965-70bcb7964dcb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:53.795200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26658fbd-23c5-4190-b965-70bcb7964dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:53.795479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26658fbd-23c5-4190-b965-70bcb7964dcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:53.800932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91c30f-b1e0-4d13-ac9c-cd8faa1bd562 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:53.808005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26658fbd-23c5-4190-b965-70bcb7964dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:53.810054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26658fbd-23c5-4190-b965-70bcb7964dcb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-26658fbd-23c5-4190-b965-70bcb7964dcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:53.824615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:53.825728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:53.826847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:53.826847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:53.826847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:53.826847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 358/725] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:53.827161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe07e07-519c-4bc4-be21-0361f0537fc9 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:53.827447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe07e07-519c-4bc4-be21-0361f0537fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:53.827721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe07e07-519c-4bc4-be21-0361f0537fc9 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:53.832345 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 359/726] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:53.832345 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe07e07-519c-4bc4-be21-0361f0537fc9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:53.847661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84502c81-869c-4271-9c9c-6822575b6907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:53.848319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84502c81-869c-4271-9c9c-6822575b6907 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:53.848869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84502c81-869c-4271-9c9c-6822575b6907 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:53.849143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84502c81-869c-4271-9c9c-6822575b6907 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:53.857268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84502c81-869c-4271-9c9c-6822575b6907 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:53.858705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84502c81-869c-4271-9c9c-6822575b6907 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-84502c81-869c-4271-9c9c-6822575b6907 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:53.869920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:53.870875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:53.871788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:53.871788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:53.871788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:53.871788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:53.871788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:53.871788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:53.871788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:53.871788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:53.871788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 368/727] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:53.872764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84502c81-869c-4271-9c9c-6822575b6907 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:53.943989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe07e07-519c-4bc4-be21-0361f0537fc9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:53.970258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:53.980154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84502c81-869c-4271-9c9c-6822575b6907 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:53.982602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbe586-24ba-4840-b8b0-c49d7794e369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:53.983260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbe586-24ba-4840-b8b0-c49d7794e369 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:53.983537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbe586-24ba-4840-b8b0-c49d7794e369 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:53.983811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbe586-24ba-4840-b8b0-c49d7794e369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:53.995587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbe586-24ba-4840-b8b0-c49d7794e369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:53.997598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbe586-24ba-4840-b8b0-c49d7794e369 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-aabbe586-24ba-4840-b8b0-c49d7794e369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:54.013697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:54.014692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:54.015585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:54.015585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:54.015585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:54.015754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752c5fa-3203-4775-a2c1-8891bb0091ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:54.015754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752c5fa-3203-4775-a2c1-8891bb0091ef None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.015754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752c5fa-3203-4775-a2c1-8891bb0091ef None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.016571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752c5fa-3203-4775-a2c1-8891bb0091ef None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.016571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752c5fa-3203-4775-a2c1-8891bb0091ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:54.016874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752c5fa-3203-4775-a2c1-8891bb0091ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:54.018241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752c5fa-3203-4775-a2c1-8891bb0091ef None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:54.019865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 360/728] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:54.021282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2556df-8907-4c8c-9f1f-865c8658ae7a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.021569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2556df-8907-4c8c-9f1f-865c8658ae7a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.021845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2556df-8907-4c8c-9f1f-865c8658ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.035526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 369/729] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:54.035526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752c5fa-3203-4775-a2c1-8891bb0091ef None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:54.038645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:54.041776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1e2556df-8907-4c8c-9f1f-865c8658ae7a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:54.042957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1e2556df-8907-4c8c-9f1f-865c8658ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:54.121004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 361/730] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:54.121004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2556df-8907-4c8c-9f1f-865c8658ae7a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:54.148127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752c5fa-3203-4775-a2c1-8891bb0091ef None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:54.187095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731e611-70b5-4e04-bef3-bb5604b3b493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:54.187753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731e611-70b5-4e04-bef3-bb5604b3b493 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.188038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731e611-70b5-4e04-bef3-bb5604b3b493 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.188335 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731e611-70b5-4e04-bef3-bb5604b3b493 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.189023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731e611-70b5-4e04-bef3-bb5604b3b493 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:54.189332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731e611-70b5-4e04-bef3-bb5604b3b493 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:54.190835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731e611-70b5-4e04-bef3-bb5604b3b493 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:54.203599 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 370/731] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:54.208993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731e611-70b5-4e04-bef3-bb5604b3b493 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:54.248823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2556df-8907-4c8c-9f1f-865c8658ae7a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:54.260366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:54.287386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c8530-9a90-420a-80ac-28e15aff7a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:54.287993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c8530-9a90-420a-80ac-28e15aff7a24 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.288317 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c8530-9a90-420a-80ac-28e15aff7a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.288598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c8530-9a90-420a-80ac-28e15aff7a24 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.289246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c8530-9a90-420a-80ac-28e15aff7a24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:54.289570 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c8530-9a90-420a-80ac-28e15aff7a24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:54.290933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c8530-9a90-420a-80ac-28e15aff7a24 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:54.299970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:54.309687 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 362/732] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:54.309687 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c8530-9a90-420a-80ac-28e15aff7a24 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:54.354069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731e611-70b5-4e04-bef3-bb5604b3b493 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:54.397204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c1c80-a4ea-455a-b593-f50262b01f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:54.398115 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c1c80-a4ea-455a-b593-f50262b01f5a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.398542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c1c80-a4ea-455a-b593-f50262b01f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.399119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c1c80-a4ea-455a-b593-f50262b01f5a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.409751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c1c80-a4ea-455a-b593-f50262b01f5a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:54.411211 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c1c80-a4ea-455a-b593-f50262b01f5a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:54.423130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c8530-9a90-420a-80ac-28e15aff7a24 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-320c1c80-a4ea-455a-b593-f50262b01f5a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:54.424403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:54.425308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 371/733] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Aug 24 09:50:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:54.426565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c1c80-a4ea-455a-b593-f50262b01f5a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:54.466893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04376fc-667f-479b-9c12-88ca1988ee01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:54.467418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04376fc-667f-479b-9c12-88ca1988ee01 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.467698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04376fc-667f-479b-9c12-88ca1988ee01 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.467973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04376fc-667f-479b-9c12-88ca1988ee01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.468735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04376fc-667f-479b-9c12-88ca1988ee01 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=98e11312493643b199fca04cb2d77853) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:54.469043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04376fc-667f-479b-9c12-88ca1988ee01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:54.470475 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04376fc-667f-479b-9c12-88ca1988ee01 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:54.476460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 363/734] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:54 2022] GET /identity/v3/role_assignments?user.id=98e11312493643b199fca04cb2d77853&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:54.477230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04376fc-667f-479b-9c12-88ca1988ee01 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:54.515087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:54.560401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c1c80-a4ea-455a-b593-f50262b01f5a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:54.590286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:54.603266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ed8ab-1668-4535-876b-656892352a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:54.604181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ed8ab-1668-4535-876b-656892352a63 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.604668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ed8ab-1668-4535-876b-656892352a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.605627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ed8ab-1668-4535-876b-656892352a63 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.614392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ed8ab-1668-4535-876b-656892352a63 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:54.616200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ed8ab-1668-4535-876b-656892352a63 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:54.622723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04376fc-667f-479b-9c12-88ca1988ee01 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-981ed8ab-1668-4535-876b-656892352a63 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:54.630146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:54.631320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:54.632411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:54.632411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:54.632411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:54.632411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:54.632411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:54.632411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:54.632411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:54.632411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:54.632411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:50:54.632411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:54.638956 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 372/735] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:54.639987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ed8ab-1668-4535-876b-656892352a63 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:54.672411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b9b4c-bab1-46a6-ad39-11f07c972bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:54.673127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b9b4c-bab1-46a6-ad39-11f07c972bc7 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.673488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b9b4c-bab1-46a6-ad39-11f07c972bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.673765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b9b4c-bab1-46a6-ad39-11f07c972bc7 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.674421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b9b4c-bab1-46a6-ad39-11f07c972bc7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:54.675806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b9b4c-bab1-46a6-ad39-11f07c972bc7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:54.695158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 364/736] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:50:54 2022] POST /identity/v3/services => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:54.695993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b9b4c-bab1-46a6-ad39-11f07c972bc7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:54.774917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ed8ab-1668-4535-876b-656892352a63 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:54.830295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7163d-a7b8-4c5f-8431-4c504e6f8de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:54.831164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7163d-a7b8-4c5f-8431-4c504e6f8de7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.831647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7163d-a7b8-4c5f-8431-4c504e6f8de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.832150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7163d-a7b8-4c5f-8431-4c504e6f8de7 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.832744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:54.833276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7163d-a7b8-4c5f-8431-4c504e6f8de7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:54.833835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7163d-a7b8-4c5f-8431-4c504e6f8de7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:54.835662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7163d-a7b8-4c5f-8431-4c504e6f8de7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:54.846373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b9b4c-bab1-46a6-ad39-11f07c972bc7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:54.858327 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 373/737] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Aug 24 09:50:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:54.859850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59579cb7-17ea-4f0c-b904-791c093f084f None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.860310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59579cb7-17ea-4f0c-b904-791c093f084f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.860757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59579cb7-17ea-4f0c-b904-791c093f084f None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.862132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 374/738] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:54.863287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59579cb7-17ea-4f0c-b904-791c093f084f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:54.886006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650de23-db50-4cda-9fe5-97179615c620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:54.886657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650de23-db50-4cda-9fe5-97179615c620 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.886937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650de23-db50-4cda-9fe5-97179615c620 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.887208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650de23-db50-4cda-9fe5-97179615c620 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.887945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650de23-db50-4cda-9fe5-97179615c620 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:54.888298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650de23-db50-4cda-9fe5-97179615c620 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:54.889829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650de23-db50-4cda-9fe5-97179615c620 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:54.909472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:54.919376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 365/739] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Aug 24 09:50:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:54.919376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d714b-4bdc-4dd5-a9de-f37b07b6f092 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:54.919376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d714b-4bdc-4dd5-a9de-f37b07b6f092 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:54.919376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d714b-4bdc-4dd5-a9de-f37b07b6f092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:54.938492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4d6d714b-4bdc-4dd5-a9de-f37b07b6f092 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:54.939102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4d6d714b-4bdc-4dd5-a9de-f37b07b6f092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:54.989843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59579cb7-17ea-4f0c-b904-791c093f084f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:55.030323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 366/740] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:50:55.031124 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d714b-4bdc-4dd5-a9de-f37b07b6f092 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:55.035492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18be7ef-25de-4738-a0c5-12f374a46e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:55.036266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18be7ef-25de-4738-a0c5-12f374a46e37 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:55.036603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18be7ef-25de-4738-a0c5-12f374a46e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:55.036883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18be7ef-25de-4738-a0c5-12f374a46e37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:55.124019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18be7ef-25de-4738-a0c5-12f374a46e37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:55.124551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:55.126562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18be7ef-25de-4738-a0c5-12f374a46e37 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:55.152969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d714b-4bdc-4dd5-a9de-f37b07b6f092 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:55.171513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:55.193849 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e46f44-9073-4863-8c6e-14204bdbfeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:55.194833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e46f44-9073-4863-8c6e-14204bdbfeab None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:55.195341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e46f44-9073-4863-8c6e-14204bdbfeab None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:55.195835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e46f44-9073-4863-8c6e-14204bdbfeab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:55.271241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e46f44-9073-4863-8c6e-14204bdbfeab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:55.273082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e46f44-9073-4863-8c6e-14204bdbfeab None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:55.306903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 375/741] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:54 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:55.312820 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18be7ef-25de-4738-a0c5-12f374a46e37 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:55.359325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:55.384242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 367/742] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:55 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:50:55.393121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e46f44-9073-4863-8c6e-14204bdbfeab None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:55.511091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18be7ef-25de-4738-a0c5-12f374a46e37 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:55.552576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faca463-d9da-404a-9eb7-8b41433c02b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:55.553238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faca463-d9da-404a-9eb7-8b41433c02b1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:55.553518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faca463-d9da-404a-9eb7-8b41433c02b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:55.553792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faca463-d9da-404a-9eb7-8b41433c02b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:55.559444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e46f44-9073-4863-8c6e-14204bdbfeab None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:55.562167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:55.570195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faca463-d9da-404a-9eb7-8b41433c02b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:55.571794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faca463-d9da-404a-9eb7-8b41433c02b1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-5faca463-d9da-404a-9eb7-8b41433c02b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:55.588712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:55.590260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:55.591174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:55.591174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:55.591174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:55.591174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 376/743] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:55.591822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faca463-d9da-404a-9eb7-8b41433c02b1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:55.599751 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7b879-1285-4d4b-b3a2-7a77aa68be9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:55.600690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7b879-1285-4d4b-b3a2-7a77aa68be9a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:55.601171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7b879-1285-4d4b-b3a2-7a77aa68be9a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:55.601646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7b879-1285-4d4b-b3a2-7a77aa68be9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:55.615408 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7b879-1285-4d4b-b3a2-7a77aa68be9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:55.617295 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7b879-1285-4d4b-b3a2-7a77aa68be9a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-20e7b879-1285-4d4b-b3a2-7a77aa68be9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:55.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:55.635339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:55.636679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:55.636679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:55.636679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:55.638547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 368/744] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:55.639556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7b879-1285-4d4b-b3a2-7a77aa68be9a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:55.673634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:55.712137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faca463-d9da-404a-9eb7-8b41433c02b1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:55.749596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7b879-1285-4d4b-b3a2-7a77aa68be9a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:55.750012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9b5fc-e268-4f78-8d1f-81b219a196fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:55.750617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9b5fc-e268-4f78-8d1f-81b219a196fb None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:55.750998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9b5fc-e268-4f78-8d1f-81b219a196fb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:55.751281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9b5fc-e268-4f78-8d1f-81b219a196fb None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:55.752031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9b5fc-e268-4f78-8d1f-81b219a196fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:55.753427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9b5fc-e268-4f78-8d1f-81b219a196fb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:55.770591 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 377/745] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:50:55 2022] POST /identity/v3/services => generated 240 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:55.771489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9b5fc-e268-4f78-8d1f-81b219a196fb None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:55.840536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:55.857338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc6971-465b-4cf9-9253-eecd417da34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:55.857963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc6971-465b-4cf9-9253-eecd417da34b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:55.858238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc6971-465b-4cf9-9253-eecd417da34b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:55.858502 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc6971-465b-4cf9-9253-eecd417da34b None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:55.859156 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc6971-465b-4cf9-9253-eecd417da34b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:55.859456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc6971-465b-4cf9-9253-eecd417da34b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:55.860884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc6971-465b-4cf9-9253-eecd417da34b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:55.895112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9b5fc-e268-4f78-8d1f-81b219a196fb None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:55.896454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 369/746] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:55.909125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc6971-465b-4cf9-9253-eecd417da34b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:55.937958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c86ce0-2728-46e6-af97-37ad4a4ad29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:55.938570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c86ce0-2728-46e6-af97-37ad4a4ad29a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:55.938851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c86ce0-2728-46e6-af97-37ad4a4ad29a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:55.939133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c86ce0-2728-46e6-af97-37ad4a4ad29a None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:55.939777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c86ce0-2728-46e6-af97-37ad4a4ad29a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:55.940108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c86ce0-2728-46e6-af97-37ad4a4ad29a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:55.941878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c86ce0-2728-46e6-af97-37ad4a4ad29a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:55.956010 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 378/747] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:55.961769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c86ce0-2728-46e6-af97-37ad4a4ad29a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:56.033855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:56.054576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc6971-465b-4cf9-9253-eecd417da34b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:56.082276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c86ce0-2728-46e6-af97-37ad4a4ad29a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:56.094574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:56.101107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd08950-3d57-496a-9806-692ba1e6184a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:56.101972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd08950-3d57-496a-9806-692ba1e6184a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:56.102248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd08950-3d57-496a-9806-692ba1e6184a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:56.102911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd08950-3d57-496a-9806-692ba1e6184a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:56.113493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd08950-3d57-496a-9806-692ba1e6184a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:56.117684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd08950-3d57-496a-9806-692ba1e6184a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:56.133432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f44f7-185c-4448-9eb5-7f7e1e7cd1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:56.134070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f44f7-185c-4448-9eb5-7f7e1e7cd1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:56.134405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f44f7-185c-4448-9eb5-7f7e1e7cd1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:56.134684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f44f7-185c-4448-9eb5-7f7e1e7cd1b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-9bd08950-3d57-496a-9806-692ba1e6184a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:56.142535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:56.143418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:56.144354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:56.144354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:56.144354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:56.144354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:56.144354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:56.144354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:56.144354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:56.144354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:56.151733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 370/748] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:56.152534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f44f7-185c-4448-9eb5-7f7e1e7cd1b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:56.154041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f44f7-185c-4448-9eb5-7f7e1e7cd1b4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:56.156019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd08950-3d57-496a-9806-692ba1e6184a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-9e9f44f7-185c-4448-9eb5-7f7e1e7cd1b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:56.176220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:56.178585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:56.180757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:56.180757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:56.180757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:56.180757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:56.180757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:56.180757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:56.180757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:56.180757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:56.180757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 379/749] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:56.185290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f44f7-185c-4448-9eb5-7f7e1e7cd1b4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:56.338045 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd08950-3d57-496a-9806-692ba1e6184a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:56.374990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f44f7-185c-4448-9eb5-7f7e1e7cd1b4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:56.405340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17985e0-5fd2-461f-b115-9a0e5d576c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:56.406004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17985e0-5fd2-461f-b115-9a0e5d576c84 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:56.406287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17985e0-5fd2-461f-b115-9a0e5d576c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:56.406561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17985e0-5fd2-461f-b115-9a0e5d576c84 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:56.407263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17985e0-5fd2-461f-b115-9a0e5d576c84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:56.407569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17985e0-5fd2-461f-b115-9a0e5d576c84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:56.409055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17985e0-5fd2-461f-b115-9a0e5d576c84 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:56.437150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 371/750] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:56.439981 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17985e0-5fd2-461f-b115-9a0e5d576c84 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:56.447332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994cbbd-d4a6-401d-ba6d-82548a8e9be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:56.447949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994cbbd-d4a6-401d-ba6d-82548a8e9be2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:56.448256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994cbbd-d4a6-401d-ba6d-82548a8e9be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:56.448528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994cbbd-d4a6-401d-ba6d-82548a8e9be2 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:56.449784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994cbbd-d4a6-401d-ba6d-82548a8e9be2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:56.450092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994cbbd-d4a6-401d-ba6d-82548a8e9be2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:56.453289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994cbbd-d4a6-401d-ba6d-82548a8e9be2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:56.475037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 380/751] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:56.480816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:56.481447 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994cbbd-d4a6-401d-ba6d-82548a8e9be2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:56.542669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:56.614842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994cbbd-d4a6-401d-ba6d-82548a8e9be2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:56.639323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17985e0-5fd2-461f-b115-9a0e5d576c84 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:56.657892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f035c35-eff5-4640-b3c2-7cb4ded6572b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:56.658677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f035c35-eff5-4640-b3c2-7cb4ded6572b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:56.658963 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f035c35-eff5-4640-b3c2-7cb4ded6572b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:56.659496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f035c35-eff5-4640-b3c2-7cb4ded6572b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:56.660371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f035c35-eff5-4640-b3c2-7cb4ded6572b None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=f1b1d85195db46b19d6a0156dbdf7f6e) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:56.661442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f035c35-eff5-4640-b3c2-7cb4ded6572b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:56.663306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f035c35-eff5-4640-b3c2-7cb4ded6572b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:56.674668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 381/752] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:56 2022] GET /identity/v3/role_assignments?user.id=f1b1d85195db46b19d6a0156dbdf7f6e&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:56.684394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2be3c-1d70-40c5-8a18-ad589fb06a5e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:56.685019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2be3c-1d70-40c5-8a18-ad589fb06a5e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:56.685521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2be3c-1d70-40c5-8a18-ad589fb06a5e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:56.687341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e7bfa-6f47-48fd-b562-ea875ebc3c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:56.687769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 382/753] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:56 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:56.692854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7bfa-6f47-48fd-b562-ea875ebc3c69 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:56.693488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7bfa-6f47-48fd-b562-ea875ebc3c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:56.693946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7bfa-6f47-48fd-b562-ea875ebc3c69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:56.694940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e7bfa-6f47-48fd-b562-ea875ebc3c69 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=1bac9dc42db9451e9e87a88c17f29c80) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:56.695470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e7bfa-6f47-48fd-b562-ea875ebc3c69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:56.697284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e7bfa-6f47-48fd-b562-ea875ebc3c69 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:56.697881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cbfce-19d7-43d9-9bc2-bda1489c17d8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:56.698235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cbfce-19d7-43d9-9bc2-bda1489c17d8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:56.698519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cbfce-19d7-43d9-9bc2-bda1489c17d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:56.708135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 372/754] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:56 2022] GET /identity/v3/role_assignments?user.id=1bac9dc42db9451e9e87a88c17f29c80&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:56.737641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fb8cbfce-19d7-43d9-9bc2-bda1489c17d8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:56.738988 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fb8cbfce-19d7-43d9-9bc2-bda1489c17d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:56.864328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 383/755] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:57.029055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746c16b-ea38-4513-8e9e-424f7e93205a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:57.029421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746c16b-ea38-4513-8e9e-424f7e93205a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:57.029624 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746c16b-ea38-4513-8e9e-424f7e93205a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:57.031109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 373/756] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:57.036646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ceaab-98b4-4774-8859-1547dfe46e90 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:57.037194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ceaab-98b4-4774-8859-1547dfe46e90 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:57.037665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ceaab-98b4-4774-8859-1547dfe46e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:57.056999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-815ceaab-98b4-4774-8859-1547dfe46e90 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:57.059124 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-815ceaab-98b4-4774-8859-1547dfe46e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:57.078094 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:57.132232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 384/757] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:57.216370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746c16b-ea38-4513-8e9e-424f7e93205a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:57.302223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:57.340558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746c16b-ea38-4513-8e9e-424f7e93205a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:57.382563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce3af5-c252-4019-8bbc-ba4f6ad20d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:57.383186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce3af5-c252-4019-8bbc-ba4f6ad20d68 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:57.383462 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce3af5-c252-4019-8bbc-ba4f6ad20d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:57.383787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce3af5-c252-4019-8bbc-ba4f6ad20d68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:57.396471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce3af5-c252-4019-8bbc-ba4f6ad20d68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-00ce3af5-c252-4019-8bbc-ba4f6ad20d68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:57.402224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:57.403677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:57.404637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 374/758] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:57.409716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ceaab-98b4-4774-8859-1547dfe46e90 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:57.501446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce3af5-c252-4019-8bbc-ba4f6ad20d68 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:57.546652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ceaab-98b4-4774-8859-1547dfe46e90 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:57.614031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdebfc8-e35c-4d33-9bcd-3a51c160203f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:57.614972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdebfc8-e35c-4d33-9bcd-3a51c160203f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:57.615428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdebfc8-e35c-4d33-9bcd-3a51c160203f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:57.615876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdebfc8-e35c-4d33-9bcd-3a51c160203f None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:57.616754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdebfc8-e35c-4d33-9bcd-3a51c160203f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:57.617320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdebfc8-e35c-4d33-9bcd-3a51c160203f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:57.619993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdebfc8-e35c-4d33-9bcd-3a51c160203f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:57.638448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 385/759] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:57.642309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdebfc8-e35c-4d33-9bcd-3a51c160203f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:57.677890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:57.692154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce3af5-c252-4019-8bbc-ba4f6ad20d68 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:57.747522 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:57.759366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac584f-a631-49ec-814f-da064dde3774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:57.760332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac584f-a631-49ec-814f-da064dde3774 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:57.760847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac584f-a631-49ec-814f-da064dde3774 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:57.761314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac584f-a631-49ec-814f-da064dde3774 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:57.762949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac584f-a631-49ec-814f-da064dde3774 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:57.772302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac584f-a631-49ec-814f-da064dde3774 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-30ac584f-a631-49ec-814f-da064dde3774 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:57.800588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:57.801630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:57.802763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 375/760] 10.222.0.86 () {56 vars in 1129 bytes} [Wed Aug 24 09:50:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:57.806005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac584f-a631-49ec-814f-da064dde3774 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:57.846962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdebfc8-e35c-4d33-9bcd-3a51c160203f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:57.909849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec20062-a5cc-4584-be43-9d449f34de37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:57.910457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec20062-a5cc-4584-be43-9d449f34de37 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:57.910691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec20062-a5cc-4584-be43-9d449f34de37 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:57.910967 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec20062-a5cc-4584-be43-9d449f34de37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:57.929325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec20062-a5cc-4584-be43-9d449f34de37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:57.932029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec20062-a5cc-4584-be43-9d449f34de37 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-2ec20062-a5cc-4584-be43-9d449f34de37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:57.955040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:57.956020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:57.956971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:57.956971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:57.956971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:57.959135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac584f-a631-49ec-814f-da064dde3774 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:57.967258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 386/761] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:57.972385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec20062-a5cc-4584-be43-9d449f34de37 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:58.008823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5987f-ae50-4434-83dc-a4af7a8e9f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:58.009729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5987f-ae50-4434-83dc-a4af7a8e9f96 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:58.010149 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5987f-ae50-4434-83dc-a4af7a8e9f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:58.010624 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5987f-ae50-4434-83dc-a4af7a8e9f96 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:58.011391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5987f-ae50-4434-83dc-a4af7a8e9f96 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:58.011888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5987f-ae50-4434-83dc-a4af7a8e9f96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:58.013420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5987f-ae50-4434-83dc-a4af7a8e9f96 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:58.018853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 376/762] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Aug 24 09:50:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:58.023916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5987f-ae50-4434-83dc-a4af7a8e9f96 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:58.073925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:58.104631 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:58.144582 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec20062-a5cc-4584-be43-9d449f34de37 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:58.151140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5987f-ae50-4434-83dc-a4af7a8e9f96 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:58.200177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b08dd-1f94-4ad3-bcf8-e5acb7ab3282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:58.200797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b08dd-1f94-4ad3-bcf8-e5acb7ab3282 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:58.201079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b08dd-1f94-4ad3-bcf8-e5acb7ab3282 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:58.201353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b08dd-1f94-4ad3-bcf8-e5acb7ab3282 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:58.202131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b08dd-1f94-4ad3-bcf8-e5acb7ab3282 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:58.202467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b08dd-1f94-4ad3-bcf8-e5acb7ab3282 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:58.204011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b08dd-1f94-4ad3-bcf8-e5acb7ab3282 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:58.205290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282a42e-c62b-485d-90ae-c1937f56ab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:58.206041 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282a42e-c62b-485d-90ae-c1937f56ab2f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:58.206455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282a42e-c62b-485d-90ae-c1937f56ab2f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:58.206934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282a42e-c62b-485d-90ae-c1937f56ab2f None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:58.207707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282a42e-c62b-485d-90ae-c1937f56ab2f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:58.208217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282a42e-c62b-485d-90ae-c1937f56ab2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:58.209826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282a42e-c62b-485d-90ae-c1937f56ab2f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:58.217268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 377/763] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Aug 24 09:50:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:58.222532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282a42e-c62b-485d-90ae-c1937f56ab2f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:58.229469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 387/764] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:58.234912 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b08dd-1f94-4ad3-bcf8-e5acb7ab3282 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:58.363283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282a42e-c62b-485d-90ae-c1937f56ab2f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:58.366524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b08dd-1f94-4ad3-bcf8-e5acb7ab3282 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:58.388075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:58.393682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:58.402423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9eeda9-5b86-4993-aa0f-b518a9df4492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:58.403725 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9eeda9-5b86-4993-aa0f-b518a9df4492 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:58.404135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9eeda9-5b86-4993-aa0f-b518a9df4492 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:58.404509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9eeda9-5b86-4993-aa0f-b518a9df4492 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:58.406842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eac501-5a4d-4620-80d3-4bb9832df96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:58.408238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac501-5a4d-4620-80d3-4bb9832df96b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:58.408660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac501-5a4d-4620-80d3-4bb9832df96b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:58.409070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac501-5a4d-4620-80d3-4bb9832df96b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:58.409909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eac501-5a4d-4620-80d3-4bb9832df96b None admin] RBAC: Adding query filter params (interface=public, service_id=82bbc83554784139bdb53ee0f140b047, region_id=RegionOne) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:58.410377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eac501-5a4d-4620-80d3-4bb9832df96b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:58.412203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eac501-5a4d-4620-80d3-4bb9832df96b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:58.412750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9eeda9-5b86-4993-aa0f-b518a9df4492 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:58.414285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9eeda9-5b86-4993-aa0f-b518a9df4492 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:58.422315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 378/765] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Aug 24 09:50:58 2022] GET /identity/v3/endpoints?service_id=82bbc83554784139bdb53ee0f140b047&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:58.423901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737820d-a02d-430f-9255-9f0a75a1aba4 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:58.424362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737820d-a02d-430f-9255-9f0a75a1aba4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:58.424793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737820d-a02d-430f-9255-9f0a75a1aba4 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:58.426196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 379/766] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:58.427657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737820d-a02d-430f-9255-9f0a75a1aba4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-fa9eeda9-5b86-4993-aa0f-b518a9df4492 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:58.431130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:58.432308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:58.433485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:58.433485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:58.433485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:58.433485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:58.433485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:58.433485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:58.433485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:58.433485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:58.433485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:50:58.433485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:58.435643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 388/767] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Aug 24 09:50:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:58.436609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866e251-1c7a-42be-8e4a-d517fd17499f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:58.436984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866e251-1c7a-42be-8e4a-d517fd17499f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:58.437350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866e251-1c7a-42be-8e4a-d517fd17499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:58.456267 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4866e251-1c7a-42be-8e4a-d517fd17499f None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:58.457514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4866e251-1c7a-42be-8e4a-d517fd17499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:58.532363 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 389/768] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:58.533409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866e251-1c7a-42be-8e4a-d517fd17499f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:58.564803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737820d-a02d-430f-9255-9f0a75a1aba4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:58.605054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9eadc-c319-4214-bebf-bff8c0d4bd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:58.606846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9eadc-c319-4214-bebf-bff8c0d4bd6b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:58.607324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9eadc-c319-4214-bebf-bff8c0d4bd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:58.608199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9eadc-c319-4214-bebf-bff8c0d4bd6b None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:58.609095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9eadc-c319-4214-bebf-bff8c0d4bd6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:58.610795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9eadc-c319-4214-bebf-bff8c0d4bd6b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:58.617784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 380/769] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Aug 24 09:50:58 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:58.700113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866e251-1c7a-42be-8e4a-d517fd17499f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:58.720201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:58.740078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002198f4-d3ff-4a76-a70c-089968c96a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:58.740702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002198f4-d3ff-4a76-a70c-089968c96a97 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:58.740979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002198f4-d3ff-4a76-a70c-089968c96a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:58.741252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002198f4-d3ff-4a76-a70c-089968c96a97 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:58.741990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002198f4-d3ff-4a76-a70c-089968c96a97 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:58.742297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002198f4-d3ff-4a76-a70c-089968c96a97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:58.743843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002198f4-d3ff-4a76-a70c-089968c96a97 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:58.770543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 390/770] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Aug 24 09:50:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:58.779023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9eadc-c319-4214-bebf-bff8c0d4bd6b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:58.845624 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:58.882293 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002198f4-d3ff-4a76-a70c-089968c96a97 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:58.909664 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9eadc-c319-4214-bebf-bff8c0d4bd6b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:58.955394 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b43085-3eb4-48f6-91a2-28d930aef038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:58.956777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b43085-3eb4-48f6-91a2-28d930aef038 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:58.957267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b43085-3eb4-48f6-91a2-28d930aef038 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:58.957733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b43085-3eb4-48f6-91a2-28d930aef038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:58.972850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b43085-3eb4-48f6-91a2-28d930aef038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:58.974960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b43085-3eb4-48f6-91a2-28d930aef038 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c2b43085-3eb4-48f6-91a2-28d930aef038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:58.994672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:58.995681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:58.997306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:58.997306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:58.997306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:59.003472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 381/771] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:59.008119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b43085-3eb4-48f6-91a2-28d930aef038 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:59.041712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002198f4-d3ff-4a76-a70c-089968c96a97 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:59.113179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b7229-c141-40fb-975f-80a4ef152053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:59.113825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b7229-c141-40fb-975f-80a4ef152053 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.114100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b7229-c141-40fb-975f-80a4ef152053 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.114375 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b7229-c141-40fb-975f-80a4ef152053 None admin] PATH_INFO: `/v3/services/network` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.114970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b7229-c141-40fb-975f-80a4ef152053 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:59.116375 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b7229-c141-40fb-975f-80a4ef152053 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-907b7229-c141-40fb-975f-80a4ef152053 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:59.128227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:59.129259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:59.130569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 391/772] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:58 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:59.131372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c29a3-7ac3-4274-961b-6c37fe7ca741 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.131372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c29a3-7ac3-4274-961b-6c37fe7ca741 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.131372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c29a3-7ac3-4274-961b-6c37fe7ca741 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.136471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 392/773] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:59.137788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c29a3-7ac3-4274-961b-6c37fe7ca741 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:59.181116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b43085-3eb4-48f6-91a2-28d930aef038 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:59.182763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:59.193432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:59.220538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6a0cc-e632-4c9f-9705-153fd6d6b4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:59.221236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6a0cc-e632-4c9f-9705-153fd6d6b4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.221518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6a0cc-e632-4c9f-9705-153fd6d6b4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.221828 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6a0cc-e632-4c9f-9705-153fd6d6b4f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.222426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6a0cc-e632-4c9f-9705-153fd6d6b4f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:59.222732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6a0cc-e632-4c9f-9705-153fd6d6b4f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:59.225178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6a0cc-e632-4c9f-9705-153fd6d6b4f0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:59.238698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 382/774] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:59.239917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b5ca5-92a7-40c2-a253-3932918193ea None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.240252 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b5ca5-92a7-40c2-a253-3932918193ea None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.240529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b5ca5-92a7-40c2-a253-3932918193ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.260464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-898b5ca5-92a7-40c2-a253-3932918193ea None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:59.261680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-898b5ca5-92a7-40c2-a253-3932918193ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:59.262824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c29a3-7ac3-4274-961b-6c37fe7ca741 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:59.297728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e539c2-c27c-4191-b01e-8c0d84967db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:59.298334 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e539c2-c27c-4191-b01e-8c0d84967db0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.298609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e539c2-c27c-4191-b01e-8c0d84967db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.298877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e539c2-c27c-4191-b01e-8c0d84967db0 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.299511 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e539c2-c27c-4191-b01e-8c0d84967db0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:59.299791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e539c2-c27c-4191-b01e-8c0d84967db0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:59.301180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e539c2-c27c-4191-b01e-8c0d84967db0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:59.307302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 393/775] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:50:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:59.308585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989150-9063-45c2-af3f-6ecc391b7325 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.308935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989150-9063-45c2-af3f-6ecc391b7325 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.309698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989150-9063-45c2-af3f-6ecc391b7325 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.311036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 394/776] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:59.315736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de989150-9063-45c2-af3f-6ecc391b7325 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:59.348207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 383/777] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:59.349165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b5ca5-92a7-40c2-a253-3932918193ea None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:59.420775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de989150-9063-45c2-af3f-6ecc391b7325 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:59.424021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:59.455454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:59.456773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999e485-f54a-4285-98e7-d37b563ef228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:59.457511 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999e485-f54a-4285-98e7-d37b563ef228 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.457937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999e485-f54a-4285-98e7-d37b563ef228 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.458307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999e485-f54a-4285-98e7-d37b563ef228 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.466001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999e485-f54a-4285-98e7-d37b563ef228 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:59.467565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999e485-f54a-4285-98e7-d37b563ef228 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:59.468792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b5ca5-92a7-40c2-a253-3932918193ea None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-f999e485-f54a-4285-98e7-d37b563ef228 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:59.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:59.480219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:59.481903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:59.481903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:59.481903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:59.481903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:59.481903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:59.481903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:59.481903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:59.481903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:50:59.487797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 395/778] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:59.489145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a52d93-9716-41eb-abb1-8da3fbaea271 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.489531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a52d93-9716-41eb-abb1-8da3fbaea271 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.489867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a52d93-9716-41eb-abb1-8da3fbaea271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.507635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-12a52d93-9716-41eb-abb1-8da3fbaea271 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:59.509052 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-12a52d93-9716-41eb-abb1-8da3fbaea271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:59.510761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807bc4c-0990-4ebf-b40c-cbf122056129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:59.511742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807bc4c-0990-4ebf-b40c-cbf122056129 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.512354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807bc4c-0990-4ebf-b40c-cbf122056129 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.512806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807bc4c-0990-4ebf-b40c-cbf122056129 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.513671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807bc4c-0990-4ebf-b40c-cbf122056129 None admin] RBAC: Adding query filter params (type=network) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:59.514201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807bc4c-0990-4ebf-b40c-cbf122056129 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:59.515853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807bc4c-0990-4ebf-b40c-cbf122056129 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:59.526537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 384/779] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:50:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:59.528259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807bc4c-0990-4ebf-b40c-cbf122056129 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:59.584181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 396/780] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:59.585217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a52d93-9716-41eb-abb1-8da3fbaea271 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:59.663596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807bc4c-0990-4ebf-b40c-cbf122056129 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:59.695480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:59.705440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a52d93-9716-41eb-abb1-8da3fbaea271 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:59.717542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66133fc-61f2-4f59-affa-b3e8d6aed28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:59.718898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66133fc-61f2-4f59-affa-b3e8d6aed28d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.719542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66133fc-61f2-4f59-affa-b3e8d6aed28d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.720005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66133fc-61f2-4f59-affa-b3e8d6aed28d None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.721179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66133fc-61f2-4f59-affa-b3e8d6aed28d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:59.722138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66133fc-61f2-4f59-affa-b3e8d6aed28d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:59.724092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66133fc-61f2-4f59-affa-b3e8d6aed28d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:59.741599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 385/781] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:59.742663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66133fc-61f2-4f59-affa-b3e8d6aed28d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:59.751075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24928a01-97e2-4df4-b9c3-5897d2456aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:59.751917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24928a01-97e2-4df4-b9c3-5897d2456aee None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.752456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24928a01-97e2-4df4-b9c3-5897d2456aee None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.752933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24928a01-97e2-4df4-b9c3-5897d2456aee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.753892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24928a01-97e2-4df4-b9c3-5897d2456aee None admin] RBAC: Adding query filter params (interface=public, service_id=fd3152acb42f443284eaa6d533fd4f81, region_id=RegionOne) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:59.754382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24928a01-97e2-4df4-b9c3-5897d2456aee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:59.756527 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24928a01-97e2-4df4-b9c3-5897d2456aee None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:59.767042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:59.772879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 397/782] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Aug 24 09:50:59 2022] GET /identity/v3/endpoints?service_id=fd3152acb42f443284eaa6d533fd4f81&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:59.773419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24928a01-97e2-4df4-b9c3-5897d2456aee None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:59.868482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66133fc-61f2-4f59-affa-b3e8d6aed28d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:59.906734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e730884-6c27-4199-8a67-d66b6aa284a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:59.907383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e730884-6c27-4199-8a67-d66b6aa284a0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.907532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24928a01-97e2-4df4-b9c3-5897d2456aee None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:59.907702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e730884-6c27-4199-8a67-d66b6aa284a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.907922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e730884-6c27-4199-8a67-d66b6aa284a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.919903 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e730884-6c27-4199-8a67-d66b6aa284a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6e730884-6c27-4199-8a67-d66b6aa284a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:59.924995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:59.925942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:50:59.926796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 386/783] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:59.927785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e730884-6c27-4199-8a67-d66b6aa284a0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:59.952130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d40c72-a7d1-4300-a872-96710b46d364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:59.952937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d40c72-a7d1-4300-a872-96710b46d364 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:59.953219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d40c72-a7d1-4300-a872-96710b46d364 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:59.953613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d40c72-a7d1-4300-a872-96710b46d364 None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/98e11312493643b199fca04cb2d77853/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:59.955858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d40c72-a7d1-4300-a872-96710b46d364 None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=98e11312493643b199fca04cb2d77853, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:59.957835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d40c72-a7d1-4300-a872-96710b46d364 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:59.980097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 398/784] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:50:59 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/98e11312493643b199fca04cb2d77853/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:50:59.980097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d40c72-a7d1-4300-a872-96710b46d364 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:00.011904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:00.027869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:00.058370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e730884-6c27-4199-8a67-d66b6aa284a0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:00.107679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0326f-ddc5-4e80-8eee-2ec9a5496535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:00.108093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d40c72-a7d1-4300-a872-96710b46d364 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:00.108916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0326f-ddc5-4e80-8eee-2ec9a5496535 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:00.109413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0326f-ddc5-4e80-8eee-2ec9a5496535 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:00.110218 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0326f-ddc5-4e80-8eee-2ec9a5496535 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:00.112138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0326f-ddc5-4e80-8eee-2ec9a5496535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:00.113843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0326f-ddc5-4e80-8eee-2ec9a5496535 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:00.122499 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 387/785] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Aug 24 09:50:59 2022] GET /identity/v3/services => generated 772 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:00.124507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0326f-ddc5-4e80-8eee-2ec9a5496535 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:00.170237 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d6337-a2b9-4c34-a6b1-333b2784e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:00.170884 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d6337-a2b9-4c34-a6b1-333b2784e725 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:00.171329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d6337-a2b9-4c34-a6b1-333b2784e725 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:00.171615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d6337-a2b9-4c34-a6b1-333b2784e725 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:00.186158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d6337-a2b9-4c34-a6b1-333b2784e725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-d23d6337-a2b9-4c34-a6b1-333b2784e725 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:00.192140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:00.193174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:00.194459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 399/786] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:00.199955 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d6337-a2b9-4c34-a6b1-333b2784e725 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:00.258278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0326f-ddc5-4e80-8eee-2ec9a5496535 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:00.294456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:00.298382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2cd34-550b-42f4-b54d-4f5bff8ed95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:00.299093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2cd34-550b-42f4-b54d-4f5bff8ed95d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:00.299696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2cd34-550b-42f4-b54d-4f5bff8ed95d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:00.299696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2cd34-550b-42f4-b54d-4f5bff8ed95d None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:00.300522 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2cd34-550b-42f4-b54d-4f5bff8ed95d None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:00.300908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2cd34-550b-42f4-b54d-4f5bff8ed95d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:00.302608 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2cd34-550b-42f4-b54d-4f5bff8ed95d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:00.312823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 388/787] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:51:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:00.320013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2cd34-550b-42f4-b54d-4f5bff8ed95d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:00.321582 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d6337-a2b9-4c34-a6b1-333b2784e725 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:00.325082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:00.365757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3bf09-10ed-46f7-b6e2-ddcbc4e365c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:00.366483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3bf09-10ed-46f7-b6e2-ddcbc4e365c2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:00.366834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3bf09-10ed-46f7-b6e2-ddcbc4e365c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:00.367179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3bf09-10ed-46f7-b6e2-ddcbc4e365c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:00.367923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3bf09-10ed-46f7-b6e2-ddcbc4e365c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:00.368324 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3bf09-10ed-46f7-b6e2-ddcbc4e365c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:00.370082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3bf09-10ed-46f7-b6e2-ddcbc4e365c2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:00.385778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 400/788] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:51:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:00.397593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3bf09-10ed-46f7-b6e2-ddcbc4e365c2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:00.461902 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2cd34-550b-42f4-b54d-4f5bff8ed95d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:00.527377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b16807-a972-4348-9c94-6d0178ec139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:00.529512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b16807-a972-4348-9c94-6d0178ec139c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:00.530167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b16807-a972-4348-9c94-6d0178ec139c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:00.531235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b16807-a972-4348-9c94-6d0178ec139c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:00.554050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b16807-a972-4348-9c94-6d0178ec139c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:00.555744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b16807-a972-4348-9c94-6d0178ec139c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-06b16807-a972-4348-9c94-6d0178ec139c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:00.584898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:00.586121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:00.587177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:00.587177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:00.587177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:00.590778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 389/789] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:00.597448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b16807-a972-4348-9c94-6d0178ec139c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:00.626708 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3bf09-10ed-46f7-b6e2-ddcbc4e365c2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:00.683461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d2265-a63b-4e8e-a3a1-1f9cd9fbcb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:00.684654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d2265-a63b-4e8e-a3a1-1f9cd9fbcb3c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:00.685118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d2265-a63b-4e8e-a3a1-1f9cd9fbcb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:00.685519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d2265-a63b-4e8e-a3a1-1f9cd9fbcb3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:00.710081 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:00.715177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d2265-a63b-4e8e-a3a1-1f9cd9fbcb3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:00.718969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d2265-a63b-4e8e-a3a1-1f9cd9fbcb3c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-474d2265-a63b-4e8e-a3a1-1f9cd9fbcb3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:00.743056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:00.744120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:00.745056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:00.745056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:00.745056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:00.752418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 401/790] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:00.755115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:00.755390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d2265-a63b-4e8e-a3a1-1f9cd9fbcb3c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:00.832868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b16807-a972-4348-9c94-6d0178ec139c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:00.867002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c82d45-a15d-45f8-b583-819b54d7e024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:00.867552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c82d45-a15d-45f8-b583-819b54d7e024 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:00.867831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c82d45-a15d-45f8-b583-819b54d7e024 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:00.868145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c82d45-a15d-45f8-b583-819b54d7e024 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:00.868786 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c82d45-a15d-45f8-b583-819b54d7e024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:00.869329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c82d45-a15d-45f8-b583-819b54d7e024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:00.870438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c82d45-a15d-45f8-b583-819b54d7e024 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:00.883323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 390/791] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:00.884728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e63cb5-15e0-4d6e-a34b-7ac94774ab52 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:00.885023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e63cb5-15e0-4d6e-a34b-7ac94774ab52 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:00.885283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e63cb5-15e0-4d6e-a34b-7ac94774ab52 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:00.886702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 391/792] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:00.891630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e63cb5-15e0-4d6e-a34b-7ac94774ab52 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:00.903466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d2265-a63b-4e8e-a3a1-1f9cd9fbcb3c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:00.939550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24df33-185f-4bdf-b9eb-e2aadb8b5512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:00.940470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24df33-185f-4bdf-b9eb-e2aadb8b5512 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:00.940895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24df33-185f-4bdf-b9eb-e2aadb8b5512 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:00.941311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24df33-185f-4bdf-b9eb-e2aadb8b5512 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:00.942095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24df33-185f-4bdf-b9eb-e2aadb8b5512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:00.942545 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24df33-185f-4bdf-b9eb-e2aadb8b5512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:00.944151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24df33-185f-4bdf-b9eb-e2aadb8b5512 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:00.960183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 402/793] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:00.961698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abf308-036d-4805-a5e5-3126cb7d186b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:00.962146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abf308-036d-4805-a5e5-3126cb7d186b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:00.962564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abf308-036d-4805-a5e5-3126cb7d186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:00.982675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-74abf308-036d-4805-a5e5-3126cb7d186b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:00.984133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-74abf308-036d-4805-a5e5-3126cb7d186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:00.999890 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e63cb5-15e0-4d6e-a34b-7ac94774ab52 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:01.014517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:01.034595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:01.038278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20af49-3d69-4c7a-9010-640f243c151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:01.039571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20af49-3d69-4c7a-9010-640f243c151e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:01.040102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20af49-3d69-4c7a-9010-640f243c151e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:01.040444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20af49-3d69-4c7a-9010-640f243c151e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:01.048953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20af49-3d69-4c7a-9010-640f243c151e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:01.050744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20af49-3d69-4c7a-9010-640f243c151e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:01.058434 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 403/794] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:01.059829 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abf308-036d-4805-a5e5-3126cb7d186b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-1c20af49-3d69-4c7a-9010-640f243c151e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:01.071067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:01.072440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:01.072440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:01.072440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:01.072440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:01.072440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:01.072440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:01.072440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:01.072440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:01.072440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:51:01.072440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:01.072440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 392/795] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Aug 24 09:51:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:01.077006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20af49-3d69-4c7a-9010-640f243c151e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:01.168023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abf308-036d-4805-a5e5-3126cb7d186b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:01.207621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d5c1e-50d4-4de9-9bc6-4587046ea1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:01.208661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d5c1e-50d4-4de9-9bc6-4587046ea1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:01.209164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d5c1e-50d4-4de9-9bc6-4587046ea1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:01.209629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d5c1e-50d4-4de9-9bc6-4587046ea1e9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:01.217579 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d5c1e-50d4-4de9-9bc6-4587046ea1e9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:01.219448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d5c1e-50d4-4de9-9bc6-4587046ea1e9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-837d5c1e-50d4-4de9-9bc6-4587046ea1e9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:01.232645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:01.233655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:01.234610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:01.234610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:01.234610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:01.234610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:01.234610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:01.234610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:01.234610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:01.234610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:01.234610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:51:01.234610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:01.242367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 404/796] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:51:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:01.243777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d5c1e-50d4-4de9-9bc6-4587046ea1e9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:01.264112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20af49-3d69-4c7a-9010-640f243c151e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:01.298583 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82987371-9e66-4bc7-ba8c-1dcd1ce5a42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:01.299239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82987371-9e66-4bc7-ba8c-1dcd1ce5a42b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:01.299507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82987371-9e66-4bc7-ba8c-1dcd1ce5a42b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:01.299776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82987371-9e66-4bc7-ba8c-1dcd1ce5a42b None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:01.302930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82987371-9e66-4bc7-ba8c-1dcd1ce5a42b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:01.303240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82987371-9e66-4bc7-ba8c-1dcd1ce5a42b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:01.304701 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82987371-9e66-4bc7-ba8c-1dcd1ce5a42b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:01.339305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:01.343680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 393/797] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Aug 24 09:51:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:01.349484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82987371-9e66-4bc7-ba8c-1dcd1ce5a42b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:01.421432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:01.438381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d5c1e-50d4-4de9-9bc6-4587046ea1e9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:01.475583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5189b59-1f24-4e1f-831d-34cceed82480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:01.476321 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5189b59-1f24-4e1f-831d-34cceed82480 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:01.476668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5189b59-1f24-4e1f-831d-34cceed82480 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:01.477005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5189b59-1f24-4e1f-831d-34cceed82480 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:01.477830 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5189b59-1f24-4e1f-831d-34cceed82480 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:01.478201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5189b59-1f24-4e1f-831d-34cceed82480 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:01.479712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5189b59-1f24-4e1f-831d-34cceed82480 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:01.503931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 405/798] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Aug 24 09:51:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:01.505113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5189b59-1f24-4e1f-831d-34cceed82480 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:01.511618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82987371-9e66-4bc7-ba8c-1dcd1ce5a42b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:01.549281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930662c-4c9e-46f1-9b6e-60e4d0c1ec7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:01.550097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930662c-4c9e-46f1-9b6e-60e4d0c1ec7f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:01.550442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930662c-4c9e-46f1-9b6e-60e4d0c1ec7f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:01.550781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930662c-4c9e-46f1-9b6e-60e4d0c1ec7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:01.563000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930662c-4c9e-46f1-9b6e-60e4d0c1ec7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:01.564945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930662c-4c9e-46f1-9b6e-60e4d0c1ec7f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5930662c-4c9e-46f1-9b6e-60e4d0c1ec7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:01.581099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:01.582975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:01.585473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:01.585473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:01.585473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:01.585473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 394/799] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:01.585473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930662c-4c9e-46f1-9b6e-60e4d0c1ec7f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:01.621161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:01.624496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5189b59-1f24-4e1f-831d-34cceed82480 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:01.663445 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939033a-79ee-4bcf-ac14-ded718cc42fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:01.664248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939033a-79ee-4bcf-ac14-ded718cc42fd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:01.664589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939033a-79ee-4bcf-ac14-ded718cc42fd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:01.664954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939033a-79ee-4bcf-ac14-ded718cc42fd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:01.665671 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939033a-79ee-4bcf-ac14-ded718cc42fd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:01.667093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939033a-79ee-4bcf-ac14-ded718cc42fd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-7939033a-79ee-4bcf-ac14-ded718cc42fd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:01.676100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:01.677300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:01.678480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 406/800] 10.222.0.86 () {56 vars in 1129 bytes} [Wed Aug 24 09:51:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:01.679261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939033a-79ee-4bcf-ac14-ded718cc42fd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:01.706538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:01.724286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930662c-4c9e-46f1-9b6e-60e4d0c1ec7f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:01.761759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f3103-0ac9-4e00-98a0-32e874d8ca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:01.762476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f3103-0ac9-4e00-98a0-32e874d8ca07 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:01.763257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f3103-0ac9-4e00-98a0-32e874d8ca07 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:01.763600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f3103-0ac9-4e00-98a0-32e874d8ca07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:01.777520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f3103-0ac9-4e00-98a0-32e874d8ca07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:01.779088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f3103-0ac9-4e00-98a0-32e874d8ca07 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-197f3103-0ac9-4e00-98a0-32e874d8ca07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:01.794798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:01.796015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:01.796925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:01.796925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:01.796925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:01.796925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 395/801] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:01.797171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f3103-0ac9-4e00-98a0-32e874d8ca07 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:01.837016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939033a-79ee-4bcf-ac14-ded718cc42fd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:01.876570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbcb55-b541-40d5-8353-b5730036991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:01.877199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbcb55-b541-40d5-8353-b5730036991e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:01.877476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbcb55-b541-40d5-8353-b5730036991e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:01.877794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbcb55-b541-40d5-8353-b5730036991e None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:01.878381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbcb55-b541-40d5-8353-b5730036991e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:01.878682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbcb55-b541-40d5-8353-b5730036991e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:01.880207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbcb55-b541-40d5-8353-b5730036991e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:01.893522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 407/802] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:01.894461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbcb55-b541-40d5-8353-b5730036991e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:01.920376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f3103-0ac9-4e00-98a0-32e874d8ca07 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:01.924420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:01.958474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb104aa-4a25-46b2-a273-ea35f902d274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:01.959125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb104aa-4a25-46b2-a273-ea35f902d274 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:01.959572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb104aa-4a25-46b2-a273-ea35f902d274 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:01.959679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb104aa-4a25-46b2-a273-ea35f902d274 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:01.960292 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb104aa-4a25-46b2-a273-ea35f902d274 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:01.960590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb104aa-4a25-46b2-a273-ea35f902d274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:01.961969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb104aa-4a25-46b2-a273-ea35f902d274 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:01.963900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:01.969700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 396/803] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Aug 24 09:51:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:01.970500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb104aa-4a25-46b2-a273-ea35f902d274 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:02.026045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbcb55-b541-40d5-8353-b5730036991e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:02.081906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb104aa-4a25-46b2-a273-ea35f902d274 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:02.123091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990949db-9bb2-4316-bf0a-0a765cf886c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:02.124162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990949db-9bb2-4316-bf0a-0a765cf886c4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.124647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990949db-9bb2-4316-bf0a-0a765cf886c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.125134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990949db-9bb2-4316-bf0a-0a765cf886c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.133793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990949db-9bb2-4316-bf0a-0a765cf886c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:02.135642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990949db-9bb2-4316-bf0a-0a765cf886c4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-990949db-9bb2-4316-bf0a-0a765cf886c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:02.151684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:02.153168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:02.154556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:02.154556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:02.154556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:02.154556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:02.154556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:02.154556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:02.154556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:02.154556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:02.157925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 397/804] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:02.159012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990949db-9bb2-4316-bf0a-0a765cf886c4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:02.190322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6e00e-c3fa-4385-9559-3237ea4ab119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:02.190920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e00e-c3fa-4385-9559-3237ea4ab119 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.191192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e00e-c3fa-4385-9559-3237ea4ab119 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.191457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e00e-c3fa-4385-9559-3237ea4ab119 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.192125 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6e00e-c3fa-4385-9559-3237ea4ab119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:02.192494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6e00e-c3fa-4385-9559-3237ea4ab119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:02.193799 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6e00e-c3fa-4385-9559-3237ea4ab119 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:02.207087 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 408/805] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:02.208709 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6e00e-c3fa-4385-9559-3237ea4ab119 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:02.241927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:02.295410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990949db-9bb2-4316-bf0a-0a765cf886c4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:02.299581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:02.318137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6e00e-c3fa-4385-9559-3237ea4ab119 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:02.332554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ba010-2be3-4e0a-bbb3-5631dd829391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:02.333390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ba010-2be3-4e0a-bbb3-5631dd829391 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.334135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ba010-2be3-4e0a-bbb3-5631dd829391 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.334135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ba010-2be3-4e0a-bbb3-5631dd829391 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.334548 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ba010-2be3-4e0a-bbb3-5631dd829391 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:02.334853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ba010-2be3-4e0a-bbb3-5631dd829391 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:02.336247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ba010-2be3-4e0a-bbb3-5631dd829391 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:02.342159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 398/806] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Aug 24 09:51:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:02.342903 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ba010-2be3-4e0a-bbb3-5631dd829391 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:02.359899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3296a-6290-431c-b74f-740fc5d8b5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:02.360529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3296a-6290-431c-b74f-740fc5d8b5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.360804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3296a-6290-431c-b74f-740fc5d8b5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.361073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3296a-6290-431c-b74f-740fc5d8b5f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.361765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3296a-6290-431c-b74f-740fc5d8b5f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:02.362069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3296a-6290-431c-b74f-740fc5d8b5f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:02.363490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3296a-6290-431c-b74f-740fc5d8b5f5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:02.374962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 409/807] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:02.376158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906afd7-f195-41eb-a51c-7b80ccc73df6 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.376470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906afd7-f195-41eb-a51c-7b80ccc73df6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.376698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906afd7-f195-41eb-a51c-7b80ccc73df6 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.378170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 410/808] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:02.378796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906afd7-f195-41eb-a51c-7b80ccc73df6 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:02.450037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ba010-2be3-4e0a-bbb3-5631dd829391 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:02.473369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:02.483929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906afd7-f195-41eb-a51c-7b80ccc73df6 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:02.484937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59984f52-29bd-43f0-a41f-bba519ecb2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:02.485561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59984f52-29bd-43f0-a41f-bba519ecb2bd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.485838 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59984f52-29bd-43f0-a41f-bba519ecb2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.486115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59984f52-29bd-43f0-a41f-bba519ecb2bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.492712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59984f52-29bd-43f0-a41f-bba519ecb2bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:02.494163 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59984f52-29bd-43f0-a41f-bba519ecb2bd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-59984f52-29bd-43f0-a41f-bba519ecb2bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:02.505677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:02.506615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:02.513260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:02.513260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:02.513260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:02.513260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:02.513260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:02.513260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:02.513260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:02.513260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:02.513260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 399/809] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:02.513260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1a6f6-08b9-49dd-bec3-6bc3845a3343 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.513260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1a6f6-08b9-49dd-bec3-6bc3845a3343 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.513881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1a6f6-08b9-49dd-bec3-6bc3845a3343 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.513881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 400/810] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:02.513881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1a6f6-08b9-49dd-bec3-6bc3845a3343 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:02.518688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e02149-365b-4980-83ed-8c9b2102f71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:02.519290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e02149-365b-4980-83ed-8c9b2102f71a None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.519567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e02149-365b-4980-83ed-8c9b2102f71a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.519835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e02149-365b-4980-83ed-8c9b2102f71a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.520537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e02149-365b-4980-83ed-8c9b2102f71a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:02.521908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e02149-365b-4980-83ed-8c9b2102f71a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:02.539114 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:02.552243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 411/811] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 09:51:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:51:02.554557 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06c349-e44e-43cf-9cba-3ed77aad93e2 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.555679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06c349-e44e-43cf-9cba-3ed77aad93e2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.556286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06c349-e44e-43cf-9cba-3ed77aad93e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.582892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-bc06c349-e44e-43cf-9cba-3ed77aad93e2 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:02.584684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bc06c349-e44e-43cf-9cba-3ed77aad93e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:02.630311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1a6f6-08b9-49dd-bec3-6bc3845a3343 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:02.660591 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 412/812] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:02.661252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06c349-e44e-43cf-9cba-3ed77aad93e2 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:02.672636 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd9790-8957-4b42-b747-feda6b85025d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:02.673314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd9790-8957-4b42-b747-feda6b85025d None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.673594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd9790-8957-4b42-b747-feda6b85025d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.673865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd9790-8957-4b42-b747-feda6b85025d None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/1bac9dc42db9451e9e87a88c17f29c80/roles/080424a21d984f0eb115d19bbd53af20` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.695336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd9790-8957-4b42-b747-feda6b85025d None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=1bac9dc42db9451e9e87a88c17f29c80, role_id=080424a21d984f0eb115d19bbd53af20)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:02.696845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd9790-8957-4b42-b747-feda6b85025d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:02.713039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 401/813] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:51:02 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/1bac9dc42db9451e9e87a88c17f29c80/roles/080424a21d984f0eb115d19bbd53af20 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:51:02.715266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4128b5-c944-49d8-b8b3-f16d8c14c977 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.715870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4128b5-c944-49d8-b8b3-f16d8c14c977 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.716348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4128b5-c944-49d8-b8b3-f16d8c14c977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.741627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2c4128b5-c944-49d8-b8b3-f16d8c14c977 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:02.743288 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2c4128b5-c944-49d8-b8b3-f16d8c14c977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:02.792592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:02.809054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06c349-e44e-43cf-9cba-3ed77aad93e2 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:02.847781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:02.855579 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fa775-4c8d-4d63-9451-8804e5756b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:02.856249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fa775-4c8d-4d63-9451-8804e5756b26 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:02.857077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fa775-4c8d-4d63-9451-8804e5756b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:02.857348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fa775-4c8d-4d63-9451-8804e5756b26 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:02.858167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fa775-4c8d-4d63-9451-8804e5756b26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:02.858685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fa775-4c8d-4d63-9451-8804e5756b26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:02.860938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fa775-4c8d-4d63-9451-8804e5756b26 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:02.867117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 402/814] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:02.877641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 413/815] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:02.883588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4128b5-c944-49d8-b8b3-f16d8c14c977 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:02.949144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fa775-4c8d-4d63-9451-8804e5756b26 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:03.012843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4128b5-c944-49d8-b8b3-f16d8c14c977 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:03.065780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cbaba-3279-4312-a9e6-83b81449eb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:03.066456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cbaba-3279-4312-a9e6-83b81449eb23 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:03.066737 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cbaba-3279-4312-a9e6-83b81449eb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:03.067012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cbaba-3279-4312-a9e6-83b81449eb23 None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/f1b1d85195db46b19d6a0156dbdf7f6e/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:03.073372 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cbaba-3279-4312-a9e6-83b81449eb23 None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=f1b1d85195db46b19d6a0156dbdf7f6e, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:03.074757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fa775-4c8d-4d63-9451-8804e5756b26 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:03.074874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cbaba-3279-4312-a9e6-83b81449eb23 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:03.107047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 403/816] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:51:02 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/f1b1d85195db46b19d6a0156dbdf7f6e/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:51:03.111857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80714bff-d69d-4506-805b-683e1f42fb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:03.112537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80714bff-d69d-4506-805b-683e1f42fb2d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:03.112818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80714bff-d69d-4506-805b-683e1f42fb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:03.113339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80714bff-d69d-4506-805b-683e1f42fb2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:03.126181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80714bff-d69d-4506-805b-683e1f42fb2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-80714bff-d69d-4506-805b-683e1f42fb2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:03.131647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:03.132707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:03.133554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 414/817] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:51:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:03.142057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cbaba-3279-4312-a9e6-83b81449eb23 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:03.178430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80714bff-d69d-4506-805b-683e1f42fb2d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:03.192702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:03.248760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:03.310281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80714bff-d69d-4506-805b-683e1f42fb2d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:03.342174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cbaba-3279-4312-a9e6-83b81449eb23 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:03.355562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47efaa-62ad-4672-a760-518b63e9db8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:03.356221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47efaa-62ad-4672-a760-518b63e9db8b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:03.356549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47efaa-62ad-4672-a760-518b63e9db8b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:03.356818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47efaa-62ad-4672-a760-518b63e9db8b None admin] PATH_INFO: `/v3/services/network` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:03.357404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47efaa-62ad-4672-a760-518b63e9db8b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:03.359013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47efaa-62ad-4672-a760-518b63e9db8b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ae47efaa-62ad-4672-a760-518b63e9db8b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:03.368623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:03.369864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:03.370760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 415/818] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:03 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:03.378811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47efaa-62ad-4672-a760-518b63e9db8b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:03.387474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6b960-bba0-4e5b-8117-64b1376d7548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:03.388751 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6b960-bba0-4e5b-8117-64b1376d7548 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:03.389030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6b960-bba0-4e5b-8117-64b1376d7548 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:03.389301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6b960-bba0-4e5b-8117-64b1376d7548 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:03.389973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6b960-bba0-4e5b-8117-64b1376d7548 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:03.390273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6b960-bba0-4e5b-8117-64b1376d7548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:03.391719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6b960-bba0-4e5b-8117-64b1376d7548 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:03.402025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 404/819] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:51:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:03.409663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6b960-bba0-4e5b-8117-64b1376d7548 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:03.532467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:03.567011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47efaa-62ad-4672-a760-518b63e9db8b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:03.569004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6b960-bba0-4e5b-8117-64b1376d7548 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:03.596772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:03.639422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b91b67-5def-47dc-b13c-0c9e39e22b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:03.640235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd193f-d28b-420e-ba17-a0a45d71d09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:03.640933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd193f-d28b-420e-ba17-a0a45d71d09d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:03.641280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd193f-d28b-420e-ba17-a0a45d71d09d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:03.641620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd193f-d28b-420e-ba17-a0a45d71d09d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:03.644640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b91b67-5def-47dc-b13c-0c9e39e22b6c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:03.644922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b91b67-5def-47dc-b13c-0c9e39e22b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:03.645192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b91b67-5def-47dc-b13c-0c9e39e22b6c None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:03.645852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b91b67-5def-47dc-b13c-0c9e39e22b6c None admin] RBAC: Adding query filter params (name=network) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:03.646155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b91b67-5def-47dc-b13c-0c9e39e22b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:03.647525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b91b67-5def-47dc-b13c-0c9e39e22b6c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:03.654567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd193f-d28b-420e-ba17-a0a45d71d09d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:03.656115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd193f-d28b-420e-ba17-a0a45d71d09d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:03.658192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 416/820] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:51:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:03.663885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b91b67-5def-47dc-b13c-0c9e39e22b6c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-71fd193f-d28b-420e-ba17-a0a45d71d09d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:03.673255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:03.674422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:03.675562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:03.675562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:03.675562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:03.676953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 405/821] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:03.682328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd193f-d28b-420e-ba17-a0a45d71d09d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:03.806667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd193f-d28b-420e-ba17-a0a45d71d09d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:03.843164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ee33a-6be1-4877-a6bf-30287b89b4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:03.843794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ee33a-6be1-4877-a6bf-30287b89b4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:03.844171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ee33a-6be1-4877-a6bf-30287b89b4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:03.844519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ee33a-6be1-4877-a6bf-30287b89b4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:03.845245 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ee33a-6be1-4877-a6bf-30287b89b4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:03.845620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ee33a-6be1-4877-a6bf-30287b89b4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:03.847063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ee33a-6be1-4877-a6bf-30287b89b4d3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:03.848668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:03.860487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 406/822] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:03.866041 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ee33a-6be1-4877-a6bf-30287b89b4d3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:03.898194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b91b67-5def-47dc-b13c-0c9e39e22b6c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:03.970413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ee33a-6be1-4877-a6bf-30287b89b4d3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:04.008021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192c1b6-ec94-4447-b05f-c2fce9a1052b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:04.008379 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5521834-1d43-4fa4-bb64-829cfcff4a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:04.008671 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192c1b6-ec94-4447-b05f-c2fce9a1052b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:04.008955 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192c1b6-ec94-4447-b05f-c2fce9a1052b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:04.009108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5521834-1d43-4fa4-bb64-829cfcff4a0c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:04.009280 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192c1b6-ec94-4447-b05f-c2fce9a1052b None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:04.009421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5521834-1d43-4fa4-bb64-829cfcff4a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:04.009758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5521834-1d43-4fa4-bb64-829cfcff4a0c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:04.009905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192c1b6-ec94-4447-b05f-c2fce9a1052b None admin] RBAC: Adding query filter params (type=network) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:04.010158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192c1b6-ec94-4447-b05f-c2fce9a1052b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:04.011552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192c1b6-ec94-4447-b05f-c2fce9a1052b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:04.018862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5521834-1d43-4fa4-bb64-829cfcff4a0c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:04.019408 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 417/823] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:51:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:04.020598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5521834-1d43-4fa4-bb64-829cfcff4a0c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:04.029696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192c1b6-ec94-4447-b05f-c2fce9a1052b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c5521834-1d43-4fa4-bb64-829cfcff4a0c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:04.034166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:04.035301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:04.036533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:04.036533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:04.036533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:04.036533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:04.036533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:04.036533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:04.036533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:04.036533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:04.036533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:51:04.036533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:04.037796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 407/824] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Aug 24 09:51:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:04.043301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5521834-1d43-4fa4-bb64-829cfcff4a0c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:04.116532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:04.151717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5521834-1d43-4fa4-bb64-829cfcff4a0c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:04.193317 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f5795-e5f6-4558-9da0-17d532bcf19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:04.194162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f5795-e5f6-4558-9da0-17d532bcf19b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:04.194520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f5795-e5f6-4558-9da0-17d532bcf19b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:04.194644 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:04.194920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f5795-e5f6-4558-9da0-17d532bcf19b None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:04.195739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f5795-e5f6-4558-9da0-17d532bcf19b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:04.196140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f5795-e5f6-4558-9da0-17d532bcf19b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:04.197848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f5795-e5f6-4558-9da0-17d532bcf19b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:04.218211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 408/825] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Aug 24 09:51:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:04.223766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f5795-e5f6-4558-9da0-17d532bcf19b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:04.310135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192c1b6-ec94-4447-b05f-c2fce9a1052b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:04.371891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1b905-5f30-45a5-8411-a3931be456ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:04.372566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b905-5f30-45a5-8411-a3931be456ab None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:04.372840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b905-5f30-45a5-8411-a3931be456ab None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:04.373106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b905-5f30-45a5-8411-a3931be456ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:04.373772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1b905-5f30-45a5-8411-a3931be456ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:04.375128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1b905-5f30-45a5-8411-a3931be456ab None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:04.410054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 418/826] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 09:51:04 2022] POST /identity/v3/endpoints => generated 337 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:51:04.412746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f5795-e5f6-4558-9da0-17d532bcf19b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:04.430174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:04.462513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb0269-cd80-409f-8411-6cd7a2d5e519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:04.463143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb0269-cd80-409f-8411-6cd7a2d5e519 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:04.463417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb0269-cd80-409f-8411-6cd7a2d5e519 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:04.463682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb0269-cd80-409f-8411-6cd7a2d5e519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:04.555114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb0269-cd80-409f-8411-6cd7a2d5e519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:04.556656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb0269-cd80-409f-8411-6cd7a2d5e519 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:04.677497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 409/827] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:04 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:51:04.682899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1b905-5f30-45a5-8411-a3931be456ab None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:04.791874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:04.797637 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1b905-5f30-45a5-8411-a3931be456ab None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:04.835411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a99ee4-0e22-4cc4-bf06-c1111288f398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:04.836036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99ee4-0e22-4cc4-bf06-c1111288f398 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:04.836383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99ee4-0e22-4cc4-bf06-c1111288f398 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:04.836656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99ee4-0e22-4cc4-bf06-c1111288f398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:04.849671 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a99ee4-0e22-4cc4-bf06-c1111288f398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:04.851191 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a99ee4-0e22-4cc4-bf06-c1111288f398 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-92a99ee4-0e22-4cc4-bf06-c1111288f398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:04.865821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:04.866806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:04.868181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:04.868181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:04.868181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:04.868181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 419/828] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:04.873108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb0269-cd80-409f-8411-6cd7a2d5e519 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:04.883536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:04.991728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb0269-cd80-409f-8411-6cd7a2d5e519 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:05.028508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2076fd-371b-4e30-b3c9-9f7f5af8c0a7 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.029022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0ab99-ef82-4d81-b33e-31f7ec67b5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:05.029394 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0ab99-ef82-4d81-b33e-31f7ec67b5bb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.029666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0ab99-ef82-4d81-b33e-31f7ec67b5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.029935 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0ab99-ef82-4d81-b33e-31f7ec67b5bb None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.030361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2076fd-371b-4e30-b3c9-9f7f5af8c0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.030583 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0ab99-ef82-4d81-b33e-31f7ec67b5bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:05.030884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0ab99-ef82-4d81-b33e-31f7ec67b5bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:05.031232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2076fd-371b-4e30-b3c9-9f7f5af8c0a7 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.032279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0ab99-ef82-4d81-b33e-31f7ec67b5bb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:05.033265 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 420/829] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:05.038572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597c8d4-c1e8-4aa5-8379-bf36ddc2e2cc None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.039083 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597c8d4-c1e8-4aa5-8379-bf36ddc2e2cc None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.039642 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597c8d4-c1e8-4aa5-8379-bf36ddc2e2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.047832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 410/830] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:05.053459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0ab99-ef82-4d81-b33e-31f7ec67b5bb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:05.061025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d597c8d4-c1e8-4aa5-8379-bf36ddc2e2cc None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:05.062483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d597c8d4-c1e8-4aa5-8379-bf36ddc2e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:05.140412 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:05.142508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 421/831] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:05.170551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0ab99-ef82-4d81-b33e-31f7ec67b5bb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:05.206427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07171d74-8edd-4230-ba46-882ef318123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:05.207104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07171d74-8edd-4230-ba46-882ef318123a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.207399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07171d74-8edd-4230-ba46-882ef318123a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.207676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07171d74-8edd-4230-ba46-882ef318123a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.214648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07171d74-8edd-4230-ba46-882ef318123a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:05.216207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07171d74-8edd-4230-ba46-882ef318123a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:05.223425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7922b-c5a2-4fad-91a3-0d7a07431830 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.224302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7922b-c5a2-4fad-91a3-0d7a07431830 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.225030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7922b-c5a2-4fad-91a3-0d7a07431830 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.227358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 422/832] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-07171d74-8edd-4230-ba46-882ef318123a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:05.227894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:05.229679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:05.230613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:05.230613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:05.230613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:05.230613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:05.230613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:05.230613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:05.230613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:05.230613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:05.230613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 411/833] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:05.233645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f19e32-6558-4117-bfcc-85e096a1ab94 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.233926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f19e32-6558-4117-bfcc-85e096a1ab94 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.234198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f19e32-6558-4117-bfcc-85e096a1ab94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.235134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07171d74-8edd-4230-ba46-882ef318123a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:05.253495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d6f19e32-6558-4117-bfcc-85e096a1ab94 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:05.254619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d6f19e32-6558-4117-bfcc-85e096a1ab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:05.336425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 423/834] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:05.358545 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07171d74-8edd-4230-ba46-882ef318123a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:05.386231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:05.395443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aeba8-def1-45a2-9e49-a21c03628c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:05.396357 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aeba8-def1-45a2-9e49-a21c03628c33 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.396843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aeba8-def1-45a2-9e49-a21c03628c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.397359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aeba8-def1-45a2-9e49-a21c03628c33 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.398265 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aeba8-def1-45a2-9e49-a21c03628c33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:05.398774 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aeba8-def1-45a2-9e49-a21c03628c33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:05.400424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aeba8-def1-45a2-9e49-a21c03628c33 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:05.413703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f19e32-6558-4117-bfcc-85e096a1ab94 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:05.414686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 412/835] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:51:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:05.421081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aeba8-def1-45a2-9e49-a21c03628c33 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:05.431730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:05.530144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aeba8-def1-45a2-9e49-a21c03628c33 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:05.531192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f19e32-6558-4117-bfcc-85e096a1ab94 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:05.568763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e6a1e-4a26-4405-a215-bdbbb2711478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:05.569778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e6a1e-4a26-4405-a215-bdbbb2711478 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.570262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e6a1e-4a26-4405-a215-bdbbb2711478 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.570731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e6a1e-4a26-4405-a215-bdbbb2711478 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.576061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44847ef-c127-457f-a7bb-7f7fc54647c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:05.576668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44847ef-c127-457f-a7bb-7f7fc54647c8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.576943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44847ef-c127-457f-a7bb-7f7fc54647c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.577212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44847ef-c127-457f-a7bb-7f7fc54647c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.577927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44847ef-c127-457f-a7bb-7f7fc54647c8 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=98e11312493643b199fca04cb2d77853) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:05.578230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44847ef-c127-457f-a7bb-7f7fc54647c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:05.579723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44847ef-c127-457f-a7bb-7f7fc54647c8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:05.582968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e6a1e-4a26-4405-a215-bdbbb2711478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:05.585987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 413/836] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:51:05 2022] GET /identity/v3/role_assignments?user.id=98e11312493643b199fca04cb2d77853&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:05.587166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff4f16-9456-466a-a685-c770034c5f89 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.587447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff4f16-9456-466a-a685-c770034c5f89 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.587715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff4f16-9456-466a-a685-c770034c5f89 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.593522 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 414/837] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-4b4e6a1e-4a26-4405-a215-bdbbb2711478 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:05.594044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:05.595252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:05.596154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 424/838] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:51:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:05.596895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4d0b4-5bd6-4b83-a72d-c8814762d93d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.596895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4d0b4-5bd6-4b83-a72d-c8814762d93d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.596895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4d0b4-5bd6-4b83-a72d-c8814762d93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.599696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e6a1e-4a26-4405-a215-bdbbb2711478 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:05.685723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:05.727528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e6a1e-4a26-4405-a215-bdbbb2711478 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:05.739013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1bf4d0b4-5bd6-4b83-a72d-c8814762d93d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:05.740259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1bf4d0b4-5bd6-4b83-a72d-c8814762d93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:05.774515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ab945-3d8c-4de3-a8e7-f74af7b1c98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:05.775125 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ab945-3d8c-4de3-a8e7-f74af7b1c98e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.775403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ab945-3d8c-4de3-a8e7-f74af7b1c98e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.775674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ab945-3d8c-4de3-a8e7-f74af7b1c98e None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.777041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ab945-3d8c-4de3-a8e7-f74af7b1c98e None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:05.777041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ab945-3d8c-4de3-a8e7-f74af7b1c98e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:05.778871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ab945-3d8c-4de3-a8e7-f74af7b1c98e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:05.786805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 425/839] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:51:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:05.787566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ab945-3d8c-4de3-a8e7-f74af7b1c98e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:05.804361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:05.825686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 415/840] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:05.826636 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4d0b4-5bd6-4b83-a72d-c8814762d93d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:05.918759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ab945-3d8c-4de3-a8e7-f74af7b1c98e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:05.944024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4d0b4-5bd6-4b83-a72d-c8814762d93d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:05.950863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:05.957769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bb712-f3de-4882-a1e6-fd8aacc81a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:05.958657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bb712-f3de-4882-a1e6-fd8aacc81a97 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.959116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bb712-f3de-4882-a1e6-fd8aacc81a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.959593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bb712-f3de-4882-a1e6-fd8aacc81a97 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.960413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bb712-f3de-4882-a1e6-fd8aacc81a97 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:05.961983 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bb712-f3de-4882-a1e6-fd8aacc81a97 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-4c7bb712-f3de-4882-a1e6-fd8aacc81a97 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:05.971264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:05.972587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:51:05.973453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:05.976204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 426/841] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:51:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:05.984205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d1053-04d6-4cb7-a51b-f2438baf4ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:05.984205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d1053-04d6-4cb7-a51b-f2438baf4ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:05.984205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d1053-04d6-4cb7-a51b-f2438baf4ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:05.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bb712-f3de-4882-a1e6-fd8aacc81a97 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:05.984592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d1053-04d6-4cb7-a51b-f2438baf4ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:05.994786 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d1053-04d6-4cb7-a51b-f2438baf4ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:05.996380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d1053-04d6-4cb7-a51b-f2438baf4ee8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6f3d1053-04d6-4cb7-a51b-f2438baf4ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:06.014562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:06.015568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:06.016521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:06.016521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:06.016521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:06.018287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 416/842] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:06.022838 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d1053-04d6-4cb7-a51b-f2438baf4ee8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:06.075027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:06.114731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bb712-f3de-4882-a1e6-fd8aacc81a97 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:06.154542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d56ed-4eb0-4f6c-b573-b66fc32445d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:06.155205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d56ed-4eb0-4f6c-b573-b66fc32445d1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:06.155494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d56ed-4eb0-4f6c-b573-b66fc32445d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:06.155776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d56ed-4eb0-4f6c-b573-b66fc32445d1 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:06.156456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d56ed-4eb0-4f6c-b573-b66fc32445d1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:06.156761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d56ed-4eb0-4f6c-b573-b66fc32445d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:06.158133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d56ed-4eb0-4f6c-b573-b66fc32445d1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:06.158505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d1053-04d6-4cb7-a51b-f2438baf4ee8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:06.168079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 427/843] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:51:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:06.168079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d56ed-4eb0-4f6c-b573-b66fc32445d1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:06.204859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd31e9-fd3b-40e4-8be0-d8bec5afd628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:06.205942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd31e9-fd3b-40e4-8be0-d8bec5afd628 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:06.205942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd31e9-fd3b-40e4-8be0-d8bec5afd628 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:06.206258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd31e9-fd3b-40e4-8be0-d8bec5afd628 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:06.206655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd31e9-fd3b-40e4-8be0-d8bec5afd628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:06.206960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd31e9-fd3b-40e4-8be0-d8bec5afd628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:06.208395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd31e9-fd3b-40e4-8be0-d8bec5afd628 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:06.225920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 417/844] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:06.225920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd31e9-fd3b-40e4-8be0-d8bec5afd628 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:06.239167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:06.284955 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d56ed-4eb0-4f6c-b573-b66fc32445d1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:06.322571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394d09f-ddfa-44fa-8544-49bbc9169fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:06.323261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394d09f-ddfa-44fa-8544-49bbc9169fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:06.323543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394d09f-ddfa-44fa-8544-49bbc9169fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:06.323821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394d09f-ddfa-44fa-8544-49bbc9169fc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:06.326017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:06.333909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd31e9-fd3b-40e4-8be0-d8bec5afd628 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:06.334659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394d09f-ddfa-44fa-8544-49bbc9169fc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-3394d09f-ddfa-44fa-8544-49bbc9169fc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:06.339682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:06.340643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:06.342022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 428/845] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:51:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:06.342735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394d09f-ddfa-44fa-8544-49bbc9169fc2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:06.374064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4adbdb-7f9e-48c3-9b71-cbc6dfd7b85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:06.374672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4adbdb-7f9e-48c3-9b71-cbc6dfd7b85d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:06.374951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4adbdb-7f9e-48c3-9b71-cbc6dfd7b85d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:06.375224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4adbdb-7f9e-48c3-9b71-cbc6dfd7b85d None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:06.375858 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4adbdb-7f9e-48c3-9b71-cbc6dfd7b85d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:06.376200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4adbdb-7f9e-48c3-9b71-cbc6dfd7b85d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:06.377566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4adbdb-7f9e-48c3-9b71-cbc6dfd7b85d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:06.382430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 418/846] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:51:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:06.383213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4adbdb-7f9e-48c3-9b71-cbc6dfd7b85d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:06.466068 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394d09f-ddfa-44fa-8544-49bbc9169fc2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:06.489042 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:06.516657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cd8d4-b2d1-43ae-bb2d-0933c8e5db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:06.517559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cd8d4-b2d1-43ae-bb2d-0933c8e5db91 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:06.518014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cd8d4-b2d1-43ae-bb2d-0933c8e5db91 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:06.518429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cd8d4-b2d1-43ae-bb2d-0933c8e5db91 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:06.528548 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cd8d4-b2d1-43ae-bb2d-0933c8e5db91 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:06.531104 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cd8d4-b2d1-43ae-bb2d-0933c8e5db91 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:06.534649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4adbdb-7f9e-48c3-9b71-cbc6dfd7b85d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c76cd8d4-b2d1-43ae-bb2d-0933c8e5db91 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:06.544959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:06.545935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:06.546902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:06.546902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:06.546902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:06.546902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:06.546902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:06.546902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:06.546902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:06.546902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:06.546902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:51:06.546902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:06.546902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 429/847] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Aug 24 09:51:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:06.553262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cd8d4-b2d1-43ae-bb2d-0933c8e5db91 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:06.572886 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac93b3-676b-4118-8f36-934b4dfa59ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:06.573492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac93b3-676b-4118-8f36-934b4dfa59ff None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:06.574025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac93b3-676b-4118-8f36-934b4dfa59ff None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:06.574025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac93b3-676b-4118-8f36-934b4dfa59ff None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:06.574686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac93b3-676b-4118-8f36-934b4dfa59ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:06.574992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac93b3-676b-4118-8f36-934b4dfa59ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:06.577864 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac93b3-676b-4118-8f36-934b4dfa59ff None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:06.587825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 419/848] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:06.594020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac93b3-676b-4118-8f36-934b4dfa59ff None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:06.620710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:06.681717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cd8d4-b2d1-43ae-bb2d-0933c8e5db91 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:06.716279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac93b3-676b-4118-8f36-934b4dfa59ff None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:06.720118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ad99e-74c5-4c22-8120-6dc0b0f955c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:06.720728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ad99e-74c5-4c22-8120-6dc0b0f955c8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:06.721009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ad99e-74c5-4c22-8120-6dc0b0f955c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:06.721282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ad99e-74c5-4c22-8120-6dc0b0f955c8 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:06.722020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ad99e-74c5-4c22-8120-6dc0b0f955c8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:06.722329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ad99e-74c5-4c22-8120-6dc0b0f955c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:06.723827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ad99e-74c5-4c22-8120-6dc0b0f955c8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:06.742788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 430/849] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Aug 24 09:51:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:06.749044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ad99e-74c5-4c22-8120-6dc0b0f955c8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:06.758132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b818b-baf9-4a8e-945d-34d0d2b360f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:06.758922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b818b-baf9-4a8e-945d-34d0d2b360f4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:06.759302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b818b-baf9-4a8e-945d-34d0d2b360f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:06.759688 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b818b-baf9-4a8e-945d-34d0d2b360f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:06.772697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b818b-baf9-4a8e-945d-34d0d2b360f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:06.776084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b818b-baf9-4a8e-945d-34d0d2b360f4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-9d2b818b-baf9-4a8e-945d-34d0d2b360f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:06.797796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:06.799010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:06.800147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:06.800147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:06.800147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:06.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:06.806751 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 420/850] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:06.807728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b818b-baf9-4a8e-945d-34d0d2b360f4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:06.874761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ad99e-74c5-4c22-8120-6dc0b0f955c8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:06.896360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:06.915280 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e48a2-dc50-4a7b-8034-3d7a48591123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:06.916178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e48a2-dc50-4a7b-8034-3d7a48591123 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:06.916606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e48a2-dc50-4a7b-8034-3d7a48591123 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:06.917026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e48a2-dc50-4a7b-8034-3d7a48591123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:06.932756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b818b-baf9-4a8e-945d-34d0d2b360f4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:06.979576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84063cbe-1a7f-47a5-b9d7-b9e362924941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:06.980488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84063cbe-1a7f-47a5-b9d7-b9e362924941 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:06.980999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84063cbe-1a7f-47a5-b9d7-b9e362924941 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:06.981422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84063cbe-1a7f-47a5-b9d7-b9e362924941 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:06.982880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84063cbe-1a7f-47a5-b9d7-b9e362924941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:06.983867 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84063cbe-1a7f-47a5-b9d7-b9e362924941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:06.985703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84063cbe-1a7f-47a5-b9d7-b9e362924941 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:07.001824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e48a2-dc50-4a7b-8034-3d7a48591123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:07.002259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 421/851] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:07.005542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e48a2-dc50-4a7b-8034-3d7a48591123 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:07.008433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84063cbe-1a7f-47a5-b9d7-b9e362924941 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:07.054587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:07.135619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 431/852] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:06 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:51:07.137260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e48a2-dc50-4a7b-8034-3d7a48591123 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:07.143027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84063cbe-1a7f-47a5-b9d7-b9e362924941 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:07.175041 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:07.181056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec4d09-4a44-43ef-9157-f1a44f3a2501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:07.181624 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4d09-4a44-43ef-9157-f1a44f3a2501 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:07.182242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4d09-4a44-43ef-9157-f1a44f3a2501 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:07.182627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4d09-4a44-43ef-9157-f1a44f3a2501 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:07.191330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec4d09-4a44-43ef-9157-f1a44f3a2501 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:07.192976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec4d09-4a44-43ef-9157-f1a44f3a2501 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0dec4d09-4a44-43ef-9157-f1a44f3a2501 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:07.206404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:07.207633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:07.208988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:07.208988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:07.208988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:07.208988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:07.208988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:07.208988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:07.208988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:07.208988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:07.208988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:51:07.208988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:07.214950 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 422/853] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:51:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:07.215958 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec4d09-4a44-43ef-9157-f1a44f3a2501 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:07.282856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e48a2-dc50-4a7b-8034-3d7a48591123 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:07.328684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec4d09-4a44-43ef-9157-f1a44f3a2501 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:07.329792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30166d1c-fe28-42c5-9cf9-561d368bab18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:07.330540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30166d1c-fe28-42c5-9cf9-561d368bab18 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:07.330957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30166d1c-fe28-42c5-9cf9-561d368bab18 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:07.331328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30166d1c-fe28-42c5-9cf9-561d368bab18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:07.340182 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:07.347744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30166d1c-fe28-42c5-9cf9-561d368bab18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:07.349406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30166d1c-fe28-42c5-9cf9-561d368bab18 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-30166d1c-fe28-42c5-9cf9-561d368bab18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:07.365386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:07.366649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:07.367554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:07.367554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:07.367554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:07.371360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 432/854] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:07.372137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30166d1c-fe28-42c5-9cf9-561d368bab18 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:07.377945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a0969-008b-4dcd-9926-bef8e94c3fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:07.378900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a0969-008b-4dcd-9926-bef8e94c3fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:07.379470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a0969-008b-4dcd-9926-bef8e94c3fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:07.379981 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a0969-008b-4dcd-9926-bef8e94c3fe7 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:07.381069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a0969-008b-4dcd-9926-bef8e94c3fe7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:07.382256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a0969-008b-4dcd-9926-bef8e94c3fe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:07.384458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a0969-008b-4dcd-9926-bef8e94c3fe7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:07.408564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 423/855] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Aug 24 09:51:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:07.415038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a0969-008b-4dcd-9926-bef8e94c3fe7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:07.489792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30166d1c-fe28-42c5-9cf9-561d368bab18 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:07.500514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:07.529039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2e67d-5041-4bfb-b1de-87abc0b28778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:07.529658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2e67d-5041-4bfb-b1de-87abc0b28778 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:07.529940 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2e67d-5041-4bfb-b1de-87abc0b28778 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:07.530216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2e67d-5041-4bfb-b1de-87abc0b28778 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:07.530879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2e67d-5041-4bfb-b1de-87abc0b28778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:07.531189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2e67d-5041-4bfb-b1de-87abc0b28778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:07.532663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2e67d-5041-4bfb-b1de-87abc0b28778 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:07.546771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 433/856] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:07.552072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2e67d-5041-4bfb-b1de-87abc0b28778 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:07.553363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a0969-008b-4dcd-9926-bef8e94c3fe7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:07.599176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46043146-0522-4bf2-9ed6-37c144de5670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:07.600216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46043146-0522-4bf2-9ed6-37c144de5670 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:07.600530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46043146-0522-4bf2-9ed6-37c144de5670 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:07.600811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46043146-0522-4bf2-9ed6-37c144de5670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:07.609876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:07.664785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2e67d-5041-4bfb-b1de-87abc0b28778 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:07.677893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46043146-0522-4bf2-9ed6-37c144de5670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:07.679595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46043146-0522-4bf2-9ed6-37c144de5670 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:07.702750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fbf62-85ca-476f-b9e0-a4b077820745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:07.703361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fbf62-85ca-476f-b9e0-a4b077820745 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:07.703635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fbf62-85ca-476f-b9e0-a4b077820745 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:07.703904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fbf62-85ca-476f-b9e0-a4b077820745 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:07.710619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fbf62-85ca-476f-b9e0-a4b077820745 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:07.712038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fbf62-85ca-476f-b9e0-a4b077820745 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-934fbf62-85ca-476f-b9e0-a4b077820745 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:07.723345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:07.724311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:07.725292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:07.725292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:07.725292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:07.725292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:07.725292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:07.725292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:07.725292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:07.725292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:07.725292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 434/857] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:07.725784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4116b-4c05-493e-bedf-1cd1c1fd8b14 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:07.726440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4116b-4c05-493e-bedf-1cd1c1fd8b14 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:07.726440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4116b-4c05-493e-bedf-1cd1c1fd8b14 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:07.731448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 435/858] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:07.731448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4116b-4c05-493e-bedf-1cd1c1fd8b14 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:07.740485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:07.796897 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 424/859] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:51:07.798210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8389958-6627-4012-9a75-0b2edf71c644 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:07.798495 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8389958-6627-4012-9a75-0b2edf71c644 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:07.798760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8389958-6627-4012-9a75-0b2edf71c644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:07.819882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b8389958-6627-4012-9a75-0b2edf71c644 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:07.821126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b8389958-6627-4012-9a75-0b2edf71c644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:07.848029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4116b-4c05-493e-bedf-1cd1c1fd8b14 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:07.860332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:07.886128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a74810-32b7-4a01-b837-91459f332f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:07.886733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a74810-32b7-4a01-b837-91459f332f17 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:07.887016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a74810-32b7-4a01-b837-91459f332f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:07.887290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a74810-32b7-4a01-b837-91459f332f17 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:07.887987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a74810-32b7-4a01-b837-91459f332f17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:07.888345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a74810-32b7-4a01-b837-91459f332f17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:07.890297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a74810-32b7-4a01-b837-91459f332f17 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:07.893114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 425/860] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:07.894037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8389958-6627-4012-9a75-0b2edf71c644 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:07.914720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 436/861] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:51:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:07.916256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a74810-32b7-4a01-b837-91459f332f17 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:07.987920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:08.009649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8389958-6627-4012-9a75-0b2edf71c644 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:08.037582 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a74810-32b7-4a01-b837-91459f332f17 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:08.047466 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750b1dd-0522-4312-a765-b8c108b6f745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:08.048191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750b1dd-0522-4312-a765-b8c108b6f745 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:08.048473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750b1dd-0522-4312-a765-b8c108b6f745 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:08.048745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750b1dd-0522-4312-a765-b8c108b6f745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:08.061132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750b1dd-0522-4312-a765-b8c108b6f745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:08.062625 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750b1dd-0522-4312-a765-b8c108b6f745 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b750b1dd-0522-4312-a765-b8c108b6f745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:08.078060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:08.079046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:08.079941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:08.079941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:08.079941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:08.081096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf9f4d-45da-4a3a-a93d-e2d23a28dd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:08.081989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf9f4d-45da-4a3a-a93d-e2d23a28dd33 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:08.082461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf9f4d-45da-4a3a-a93d-e2d23a28dd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:08.082936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf9f4d-45da-4a3a-a93d-e2d23a28dd33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:08.083920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf9f4d-45da-4a3a-a93d-e2d23a28dd33 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=1bac9dc42db9451e9e87a88c17f29c80) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:08.084087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 426/862] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:08.084707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf9f4d-45da-4a3a-a93d-e2d23a28dd33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:08.085178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750b1dd-0522-4312-a765-b8c108b6f745 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:08.086705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf9f4d-45da-4a3a-a93d-e2d23a28dd33 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:08.100228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 437/863] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:51:07 2022] GET /identity/v3/role_assignments?user.id=1bac9dc42db9451e9e87a88c17f29c80&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:08.163766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf9f4d-45da-4a3a-a93d-e2d23a28dd33 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:08.201021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750b1dd-0522-4312-a765-b8c108b6f745 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:08.216110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:08.237334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:08.243949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cc306-cd57-44b9-a534-5dee9eb93975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:08.244599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cc306-cd57-44b9-a534-5dee9eb93975 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:08.244878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cc306-cd57-44b9-a534-5dee9eb93975 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:08.245217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cc306-cd57-44b9-a534-5dee9eb93975 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:08.245884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6cc306-cd57-44b9-a534-5dee9eb93975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:08.246190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6cc306-cd57-44b9-a534-5dee9eb93975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:08.247585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6cc306-cd57-44b9-a534-5dee9eb93975 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:08.266455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 427/864] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:08.267800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cc306-cd57-44b9-a534-5dee9eb93975 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:08.288745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf9f4d-45da-4a3a-a93d-e2d23a28dd33 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:08.331938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fb0de-283f-4481-9368-ffac683eeb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:08.332751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb0de-283f-4481-9368-ffac683eeb03 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:08.333033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb0de-283f-4481-9368-ffac683eeb03 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:08.333306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb0de-283f-4481-9368-ffac683eeb03 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:08.333893 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fb0de-283f-4481-9368-ffac683eeb03 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:08.335271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fb0de-283f-4481-9368-ffac683eeb03 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a99fb0de-283f-4481-9368-ffac683eeb03 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:08.347206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:08.348229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:08.349135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 438/865] 10.222.0.86 () {56 vars in 1132 bytes} [Wed Aug 24 09:51:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:08.354447 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fb0de-283f-4481-9368-ffac683eeb03 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:08.399476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cc306-cd57-44b9-a534-5dee9eb93975 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:08.437666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277f1e9-f21f-457c-a730-ac6ace88e2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:08.438293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277f1e9-f21f-457c-a730-ac6ace88e2cf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:08.438573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277f1e9-f21f-457c-a730-ac6ace88e2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:08.438846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277f1e9-f21f-457c-a730-ac6ace88e2cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:08.445971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277f1e9-f21f-457c-a730-ac6ace88e2cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:08.447413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277f1e9-f21f-457c-a730-ac6ace88e2cf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4277f1e9-f21f-457c-a730-ac6ace88e2cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:08.458850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:08.459916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:08.461325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:08.461325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:08.461325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:08.461325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:08.461325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:08.461325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:08.461325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:08.461325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:08.461325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 428/866] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:08.461325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b5db0-633b-4e6e-ad9c-865979b6b548 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:08.461872 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b5db0-633b-4e6e-ad9c-865979b6b548 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:08.461872 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b5db0-633b-4e6e-ad9c-865979b6b548 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:08.467462 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 429/867] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:08.467462 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b5db0-633b-4e6e-ad9c-865979b6b548 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:08.472877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:08.477606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fb0de-283f-4481-9368-ffac683eeb03 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:08.512320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef1db9-e893-4db1-84af-c7b9f736f8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:08.512927 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef1db9-e893-4db1-84af-c7b9f736f8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:08.513207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef1db9-e893-4db1-84af-c7b9f736f8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:08.513512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef1db9-e893-4db1-84af-c7b9f736f8e9 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:08.514143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef1db9-e893-4db1-84af-c7b9f736f8e9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:08.514449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef1db9-e893-4db1-84af-c7b9f736f8e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:08.515816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef1db9-e893-4db1-84af-c7b9f736f8e9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:08.525213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 439/868] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Aug 24 09:51:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:08.525213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb99d4-9e57-4e06-9833-a387ff784691 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:08.525213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb99d4-9e57-4e06-9833-a387ff784691 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:08.525213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb99d4-9e57-4e06-9833-a387ff784691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:08.534532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:08.542713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0ffb99d4-9e57-4e06-9833-a387ff784691 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:08.544795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0ffb99d4-9e57-4e06-9833-a387ff784691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:08.588951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b5db0-633b-4e6e-ad9c-865979b6b548 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:08.618397 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 440/869] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:08.625289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb99d4-9e57-4e06-9833-a387ff784691 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:08.626190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba787800-9142-42d8-92fa-9ae86cab7e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:08.626788 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba787800-9142-42d8-92fa-9ae86cab7e2d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:08.627072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba787800-9142-42d8-92fa-9ae86cab7e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:08.627344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba787800-9142-42d8-92fa-9ae86cab7e2d None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:08.628032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba787800-9142-42d8-92fa-9ae86cab7e2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:08.628367 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba787800-9142-42d8-92fa-9ae86cab7e2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:08.629790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba787800-9142-42d8-92fa-9ae86cab7e2d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:08.647316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 430/870] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:08.648415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba787800-9142-42d8-92fa-9ae86cab7e2d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:08.744001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:08.754460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb99d4-9e57-4e06-9833-a387ff784691 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:08.778433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba787800-9142-42d8-92fa-9ae86cab7e2d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:08.793212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35926a9-a097-492c-88cc-68e48450321b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:08.794141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35926a9-a097-492c-88cc-68e48450321b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:08.794636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35926a9-a097-492c-88cc-68e48450321b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:08.795135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35926a9-a097-492c-88cc-68e48450321b None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:08.795544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:08.796238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35926a9-a097-492c-88cc-68e48450321b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:08.796735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35926a9-a097-492c-88cc-68e48450321b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:08.798504 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35926a9-a097-492c-88cc-68e48450321b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:08.809788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 441/871] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Aug 24 09:51:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:08.820239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9e100-f55e-4b88-862c-f5db8afe45ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:08.820854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9e100-f55e-4b88-862c-f5db8afe45ee None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:08.821189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9e100-f55e-4b88-862c-f5db8afe45ee None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:08.821467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9e100-f55e-4b88-862c-f5db8afe45ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:08.822211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9e100-f55e-4b88-862c-f5db8afe45ee None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=f1b1d85195db46b19d6a0156dbdf7f6e) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:08.822518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9e100-f55e-4b88-862c-f5db8afe45ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:08.823990 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9e100-f55e-4b88-862c-f5db8afe45ee None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:08.831843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 431/872] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:51:08 2022] GET /identity/v3/role_assignments?user.id=f1b1d85195db46b19d6a0156dbdf7f6e&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:08.890187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35926a9-a097-492c-88cc-68e48450321b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:09.015338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35926a9-a097-492c-88cc-68e48450321b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:09.052356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bb6b0-fddf-44c7-a7d7-8d6267b03dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:09.053073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb6b0-fddf-44c7-a7d7-8d6267b03dd0 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:09.053429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb6b0-fddf-44c7-a7d7-8d6267b03dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:09.053703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb6b0-fddf-44c7-a7d7-8d6267b03dd0 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:09.054623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bb6b0-fddf-44c7-a7d7-8d6267b03dd0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:09.055996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bb6b0-fddf-44c7-a7d7-8d6267b03dd0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:09.072225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 442/873] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:51:08 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:51:10.198199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f67416-1eee-4d3b-a50e-680f771bfda6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:10.198754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f67416-1eee-4d3b-a50e-680f771bfda6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:10.198754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f67416-1eee-4d3b-a50e-680f771bfda6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:10.200395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 432/874] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:10.206286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fed1e-e5a6-4519-993f-e76d05191e2f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:10.206669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fed1e-e5a6-4519-993f-e76d05191e2f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:10.206852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fed1e-e5a6-4519-993f-e76d05191e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:10.225740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-dd4fed1e-e5a6-4519-993f-e76d05191e2f None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:10.226991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dd4fed1e-e5a6-4519-993f-e76d05191e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:10.239020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:10.308186 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 443/875] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:10 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:10.576470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f67416-1eee-4d3b-a50e-680f771bfda6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:10.689297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f67416-1eee-4d3b-a50e-680f771bfda6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:10.724405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695353b-b9d4-449e-993d-0f2314d82a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:10.725098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695353b-b9d4-449e-993d-0f2314d82a05 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:10.725382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695353b-b9d4-449e-993d-0f2314d82a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:10.725652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695353b-b9d4-449e-993d-0f2314d82a05 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:10.738563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695353b-b9d4-449e-993d-0f2314d82a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6695353b-b9d4-449e-993d-0f2314d82a05 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:10.743153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:10.744102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:10.744940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 433/876] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Aug 24 09:51:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:10.753377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fed1e-e5a6-4519-993f-e76d05191e2f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:10.793053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2346253-b932-43a3-9d1a-f6b0e2aac278 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:10.793463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2346253-b932-43a3-9d1a-f6b0e2aac278 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:10.793618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2346253-b932-43a3-9d1a-f6b0e2aac278 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:10.795796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 434/877] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:10.801231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff195c-5425-4e89-bb33-18c57b134e28 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:10.801513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff195c-5425-4e89-bb33-18c57b134e28 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:10.801790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff195c-5425-4e89-bb33-18c57b134e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:10.821447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-eaff195c-5425-4e89-bb33-18c57b134e28 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:10.822640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eaff195c-5425-4e89-bb33-18c57b134e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:10.855329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:10.874476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fed1e-e5a6-4519-993f-e76d05191e2f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:10.893290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 435/878] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:10 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:10.908512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f21bf-14de-476c-a799-e2cc6a990f4f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:10.908882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f21bf-14de-476c-a799-e2cc6a990f4f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:10.909262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f21bf-14de-476c-a799-e2cc6a990f4f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:10.911017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 436/879] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:10.916426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7e60-5db9-4a3b-8fc2-589ebb382fa5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:10.916873 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7e60-5db9-4a3b-8fc2-589ebb382fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:10.917287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7e60-5db9-4a3b-8fc2-589ebb382fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:10.923953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f1fac-14db-458d-bf7c-8aca9c83ebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:10.924685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f1fac-14db-458d-bf7c-8aca9c83ebcc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:10.925059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f1fac-14db-458d-bf7c-8aca9c83ebcc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:10.925395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f1fac-14db-458d-bf7c-8aca9c83ebcc None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:10.926124 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f1fac-14db-458d-bf7c-8aca9c83ebcc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:10.926494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f1fac-14db-458d-bf7c-8aca9c83ebcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:10.927947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f1fac-14db-458d-bf7c-8aca9c83ebcc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:10.937677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-464b7e60-5db9-4a3b-8fc2-589ebb382fa5 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:10.939129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-464b7e60-5db9-4a3b-8fc2-589ebb382fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:10.941824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 444/880] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Aug 24 09:51:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:10.943977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f1fac-14db-458d-bf7c-8aca9c83ebcc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:10.947766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:11.024508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 437/881] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:10 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:11.048470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe012b-3963-4b09-8575-edfbc8ed2326 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:11.048818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe012b-3963-4b09-8575-edfbc8ed2326 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:11.049040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe012b-3963-4b09-8575-edfbc8ed2326 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:11.050654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 438/882] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:11.056523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47188ba5-410f-4e26-9d96-d5ed190bffb8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:11.056809 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47188ba5-410f-4e26-9d96-d5ed190bffb8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:11.057090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47188ba5-410f-4e26-9d96-d5ed190bffb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:11.075406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f1fac-14db-458d-bf7c-8aca9c83ebcc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:11.075606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-47188ba5-410f-4e26-9d96-d5ed190bffb8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:11.077275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-47188ba5-410f-4e26-9d96-d5ed190bffb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:11.118666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbddb6-0619-478a-b788-1abf815a599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:11.120131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbddb6-0619-478a-b788-1abf815a599d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:11.120419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbddb6-0619-478a-b788-1abf815a599d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:11.120694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbddb6-0619-478a-b788-1abf815a599d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:11.134966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbddb6-0619-478a-b788-1abf815a599d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:11.136877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbddb6-0619-478a-b788-1abf815a599d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-aefbddb6-0619-478a-b788-1abf815a599d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:11.157128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:11.158360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:11.159491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:11.159491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:11.159491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:11.159686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 439/883] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:11.167117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 445/884] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:11.176965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47188ba5-410f-4e26-9d96-d5ed190bffb8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:11.208063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbddb6-0619-478a-b788-1abf815a599d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:11.210970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:11.282574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:11.316033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47188ba5-410f-4e26-9d96-d5ed190bffb8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:11.343287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbddb6-0619-478a-b788-1abf815a599d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:11.364520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972a7aa-0795-4699-b639-a3d7f84182b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:11.365255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972a7aa-0795-4699-b639-a3d7f84182b5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:11.365536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972a7aa-0795-4699-b639-a3d7f84182b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:11.365807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972a7aa-0795-4699-b639-a3d7f84182b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:11.368366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972a7aa-0795-4699-b639-a3d7f84182b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:11.368366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972a7aa-0795-4699-b639-a3d7f84182b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:11.368366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972a7aa-0795-4699-b639-a3d7f84182b5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:11.385787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 440/885] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:11.385787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972a7aa-0795-4699-b639-a3d7f84182b5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:11.391545 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f3597-99ec-4d15-98ea-359ad423799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:11.392603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f3597-99ec-4d15-98ea-359ad423799b None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:11.392890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f3597-99ec-4d15-98ea-359ad423799b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:11.393164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f3597-99ec-4d15-98ea-359ad423799b None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:11.393830 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f3597-99ec-4d15-98ea-359ad423799b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:11.395199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f3597-99ec-4d15-98ea-359ad423799b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:11.424100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 446/886] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:51:11 2022] POST /identity/v3/services => generated 269 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:51:11.424100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f3597-99ec-4d15-98ea-359ad423799b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:11.521073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:11.554036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972a7aa-0795-4699-b639-a3d7f84182b5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:11.599723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f3597-99ec-4d15-98ea-359ad423799b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:11.605620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ccd23-ba9f-418e-8686-70a5ead6931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:11.606254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ccd23-ba9f-418e-8686-70a5ead6931e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:11.606530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ccd23-ba9f-418e-8686-70a5ead6931e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:11.606800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ccd23-ba9f-418e-8686-70a5ead6931e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:11.607390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ccd23-ba9f-418e-8686-70a5ead6931e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:11.608876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ccd23-ba9f-418e-8686-70a5ead6931e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:11.615668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7f0ccd23-ba9f-418e-8686-70a5ead6931e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:11.620679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:11.621824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:11.622725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 441/887] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Aug 24 09:51:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:11.624480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ccd23-ba9f-418e-8686-70a5ead6931e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:11.642210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b043cde-9a1b-48fe-a0f1-cd8387b7c4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:11.642871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b043cde-9a1b-48fe-a0f1-cd8387b7c4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:11.643221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b043cde-9a1b-48fe-a0f1-cd8387b7c4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:11.643499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b043cde-9a1b-48fe-a0f1-cd8387b7c4b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:11.652832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b043cde-9a1b-48fe-a0f1-cd8387b7c4b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:11.654882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b043cde-9a1b-48fe-a0f1-cd8387b7c4b3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-8b043cde-9a1b-48fe-a0f1-cd8387b7c4b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:11.666733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:11.667910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:11.669126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:11.669126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:11.669126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:11.669126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:11.669126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:11.669126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:11.669126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:11.669126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:11.669126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:51:11.669126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:11.669990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 447/888] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Aug 24 09:51:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:11.671253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b043cde-9a1b-48fe-a0f1-cd8387b7c4b3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:11.750679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ccd23-ba9f-418e-8686-70a5ead6931e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:11.775768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b043cde-9a1b-48fe-a0f1-cd8387b7c4b3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:11.792938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91399ee-44ac-41ff-b3aa-34050820a959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:11.793775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91399ee-44ac-41ff-b3aa-34050820a959 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:11.794058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91399ee-44ac-41ff-b3aa-34050820a959 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:11.794405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91399ee-44ac-41ff-b3aa-34050820a959 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:11.795061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91399ee-44ac-41ff-b3aa-34050820a959 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:11.797060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91399ee-44ac-41ff-b3aa-34050820a959 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c91399ee-44ac-41ff-b3aa-34050820a959 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:11.805909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:11.806911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:11.807760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 442/889] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:51:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:11.809231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91399ee-44ac-41ff-b3aa-34050820a959 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:11.818373 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952bc89-a74f-45b9-b2b9-53ddd2c67b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:11.819062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952bc89-a74f-45b9-b2b9-53ddd2c67b60 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:11.819340 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952bc89-a74f-45b9-b2b9-53ddd2c67b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:11.819606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952bc89-a74f-45b9-b2b9-53ddd2c67b60 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:11.820268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952bc89-a74f-45b9-b2b9-53ddd2c67b60 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:11.820567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952bc89-a74f-45b9-b2b9-53ddd2c67b60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:11.821927 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952bc89-a74f-45b9-b2b9-53ddd2c67b60 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:11.826837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 448/890] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Aug 24 09:51:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:11.827552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952bc89-a74f-45b9-b2b9-53ddd2c67b60 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:11.834032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:11.866560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:11.922432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91399ee-44ac-41ff-b3aa-34050820a959 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:11.938101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952bc89-a74f-45b9-b2b9-53ddd2c67b60 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:11.961033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7743a-b132-4c54-a01f-ba7583219e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:11.961640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7743a-b132-4c54-a01f-ba7583219e65 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:11.961907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7743a-b132-4c54-a01f-ba7583219e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:11.962183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7743a-b132-4c54-a01f-ba7583219e65 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:11.962915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7743a-b132-4c54-a01f-ba7583219e65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:11.963222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7743a-b132-4c54-a01f-ba7583219e65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:11.964681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7743a-b132-4c54-a01f-ba7583219e65 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:11.974952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427c0d4-d4c7-4889-a5e4-ad8837638109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:11.975554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427c0d4-d4c7-4889-a5e4-ad8837638109 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:11.975834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427c0d4-d4c7-4889-a5e4-ad8837638109 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:11.976138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427c0d4-d4c7-4889-a5e4-ad8837638109 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:11.976767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427c0d4-d4c7-4889-a5e4-ad8837638109 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:11.977069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427c0d4-d4c7-4889-a5e4-ad8837638109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:11.978499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427c0d4-d4c7-4889-a5e4-ad8837638109 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:11.983316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 449/891] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:51:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:11.984117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427c0d4-d4c7-4889-a5e4-ad8837638109 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:11.984679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 443/892] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Aug 24 09:51:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:11.988763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7743a-b132-4c54-a01f-ba7583219e65 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:12.066365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:12.096684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:12.098705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7743a-b132-4c54-a01f-ba7583219e65 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:12.100275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427c0d4-d4c7-4889-a5e4-ad8837638109 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:12.137671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3cd64-ab32-414b-8800-c3dc8d41c23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:12.138278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3cd64-ab32-414b-8800-c3dc8d41c23c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:12.138589 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3cd64-ab32-414b-8800-c3dc8d41c23c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:12.138826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3cd64-ab32-414b-8800-c3dc8d41c23c None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:12.139853 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832af06-450e-468a-8b99-257f173d9bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:12.140034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3cd64-ab32-414b-8800-c3dc8d41c23c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:12.140360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3cd64-ab32-414b-8800-c3dc8d41c23c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:12.140530 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832af06-450e-468a-8b99-257f173d9bbd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:12.140756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832af06-450e-468a-8b99-257f173d9bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:12.141024 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832af06-450e-468a-8b99-257f173d9bbd None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:12.141646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832af06-450e-468a-8b99-257f173d9bbd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:12.141737 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3cd64-ab32-414b-8800-c3dc8d41c23c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:12.141947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832af06-450e-468a-8b99-257f173d9bbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:12.143293 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832af06-450e-468a-8b99-257f173d9bbd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:12.152581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 444/893] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:51:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:12.152581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3cd64-ab32-414b-8800-c3dc8d41c23c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:12.156952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 450/894] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Aug 24 09:51:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:12.157525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832af06-450e-468a-8b99-257f173d9bbd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:12.281058 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832af06-450e-468a-8b99-257f173d9bbd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:12.284081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3cd64-ab32-414b-8800-c3dc8d41c23c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:12.320134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b68ba-5fc7-44ee-8e2a-6e4699375128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:12.320748 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b68ba-5fc7-44ee-8e2a-6e4699375128 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:12.321031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b68ba-5fc7-44ee-8e2a-6e4699375128 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:12.321313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b68ba-5fc7-44ee-8e2a-6e4699375128 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:12.322023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b68ba-5fc7-44ee-8e2a-6e4699375128 None admin] RBAC: Adding query filter params (interface=public, service_id=d63c9ffba9084ccd973d97c7e113513e, region_id=RegionOne) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:12.322337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b68ba-5fc7-44ee-8e2a-6e4699375128 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:12.323710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b68ba-5fc7-44ee-8e2a-6e4699375128 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:12.330408 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 451/895] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Aug 24 09:51:12 2022] GET /identity/v3/endpoints?service_id=d63c9ffba9084ccd973d97c7e113513e&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:12.330562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ad8e7-b462-44ea-9cc6-cd431e461387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:12.330918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ad8e7-b462-44ea-9cc6-cd431e461387 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:12.331198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ad8e7-b462-44ea-9cc6-cd431e461387 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:12.331472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ad8e7-b462-44ea-9cc6-cd431e461387 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:12.340202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b68ba-5fc7-44ee-8e2a-6e4699375128 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:12.341309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:12.382648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:12.427399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ad8e7-b462-44ea-9cc6-cd431e461387 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:12.429083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ad8e7-b462-44ea-9cc6-cd431e461387 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:12.481754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b68ba-5fc7-44ee-8e2a-6e4699375128 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:12.529119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09cddd-fa24-403e-9495-278af4c1d187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:12.530869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09cddd-fa24-403e-9495-278af4c1d187 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:12.531548 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09cddd-fa24-403e-9495-278af4c1d187 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:12.531863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09cddd-fa24-403e-9495-278af4c1d187 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:12.532760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09cddd-fa24-403e-9495-278af4c1d187 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:12.534640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09cddd-fa24-403e-9495-278af4c1d187 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:12.543578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 452/896] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Aug 24 09:51:12 2022] GET /identity/v3/services => generated 1268 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:51:12.563671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 445/897] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:12 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:51:12.569801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09cddd-fa24-403e-9495-278af4c1d187 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:12.686009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:12.697086 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09cddd-fa24-403e-9495-278af4c1d187 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:12.734951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a089b-856a-45ea-a4cb-4f3ee94ff1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:12.735770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a089b-856a-45ea-a4cb-4f3ee94ff1de None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:12.736180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a089b-856a-45ea-a4cb-4f3ee94ff1de None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:12.736551 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a089b-856a-45ea-a4cb-4f3ee94ff1de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:12.747980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a089b-856a-45ea-a4cb-4f3ee94ff1de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:12.749603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a089b-856a-45ea-a4cb-4f3ee94ff1de None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a50a089b-856a-45ea-a4cb-4f3ee94ff1de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:12.764839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:12.766013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:12.767140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:12.767140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:12.767140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:12.768456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 453/898] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:12.775432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ad8e7-b462-44ea-9cc6-cd431e461387 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:12.817402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:12.886884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ad8e7-b462-44ea-9cc6-cd431e461387 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:12.922440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9206ed2-0494-470b-a684-a677399bb512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:12.923049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206ed2-0494-470b-a684-a677399bb512 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:12.923329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206ed2-0494-470b-a684-a677399bb512 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:12.923599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206ed2-0494-470b-a684-a677399bb512 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:12.924260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9206ed2-0494-470b-a684-a677399bb512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:12.924565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9206ed2-0494-470b-a684-a677399bb512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:12.925918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9206ed2-0494-470b-a684-a677399bb512 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:12.941888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 446/899] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:12.944136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a089b-856a-45ea-a4cb-4f3ee94ff1de None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:13.047244 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a089b-856a-45ea-a4cb-4f3ee94ff1de None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:13.084956 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14938f9-3729-4122-b29d-5d6aa0d2e05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:13.085508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14938f9-3729-4122-b29d-5d6aa0d2e05f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:13.085785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14938f9-3729-4122-b29d-5d6aa0d2e05f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:13.086112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14938f9-3729-4122-b29d-5d6aa0d2e05f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:13.093243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14938f9-3729-4122-b29d-5d6aa0d2e05f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:13.094692 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14938f9-3729-4122-b29d-5d6aa0d2e05f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a14938f9-3729-4122-b29d-5d6aa0d2e05f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:13.106146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:13.107091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:13.108123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:13.108123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:13.108123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:13.108123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:13.108123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:13.108123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:13.108123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:13.108123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:13.112068 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 454/900] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:13.113409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9206ed2-0494-470b-a684-a677399bb512 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:13.214757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:13.229995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9206ed2-0494-470b-a684-a677399bb512 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:13.266315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d2264-f16b-41d6-9985-cd7e9bc3df50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:13.266913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d2264-f16b-41d6-9985-cd7e9bc3df50 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:13.267189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d2264-f16b-41d6-9985-cd7e9bc3df50 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:13.267458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d2264-f16b-41d6-9985-cd7e9bc3df50 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:13.268170 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d2264-f16b-41d6-9985-cd7e9bc3df50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:13.268473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d2264-f16b-41d6-9985-cd7e9bc3df50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:13.269903 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d2264-f16b-41d6-9985-cd7e9bc3df50 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:13.283802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 447/901] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:51:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:13.289825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14938f9-3729-4122-b29d-5d6aa0d2e05f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:13.294307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:13.400568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14938f9-3729-4122-b29d-5d6aa0d2e05f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:13.436186 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25290abe-4330-4f5d-aab6-315c4ad7c9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:13.437200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25290abe-4330-4f5d-aab6-315c4ad7c9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:13.437680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25290abe-4330-4f5d-aab6-315c4ad7c9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:13.438216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25290abe-4330-4f5d-aab6-315c4ad7c9d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:13.439193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25290abe-4330-4f5d-aab6-315c4ad7c9d3 None admin] RBAC: Adding query filter params (role.id=641561cf524144e5a49f595528477389, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=1bac9dc42db9451e9e87a88c17f29c80) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:13.439703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25290abe-4330-4f5d-aab6-315c4ad7c9d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:13.441367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25290abe-4330-4f5d-aab6-315c4ad7c9d3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:13.452316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 455/902] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:51:13 2022] GET /identity/v3/role_assignments?user.id=1bac9dc42db9451e9e87a88c17f29c80&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=641561cf524144e5a49f595528477389 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:13.479136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697bda9-a1a9-4355-930d-35b67a48d4c8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:13.479649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697bda9-a1a9-4355-930d-35b67a48d4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:13.479725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697bda9-a1a9-4355-930d-35b67a48d4c8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:13.481379 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 448/903] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:13.487348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d29000-1bba-49e2-b773-85de4d353c97 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:13.487641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d29000-1bba-49e2-b773-85de4d353c97 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:13.487904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d29000-1bba-49e2-b773-85de4d353c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:13.507656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-53d29000-1bba-49e2-b773-85de4d353c97 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:13.508925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-53d29000-1bba-49e2-b773-85de4d353c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:13.580182 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:13.587769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 456/904] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:13.856375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697bda9-a1a9-4355-930d-35b67a48d4c8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:13.964342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697bda9-a1a9-4355-930d-35b67a48d4c8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:14.001518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a7df8-a9c3-41ae-b1f7-658d7bf66fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:14.002532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a7df8-a9c3-41ae-b1f7-658d7bf66fcc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:14.003012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a7df8-a9c3-41ae-b1f7-658d7bf66fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:14.003481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a7df8-a9c3-41ae-b1f7-658d7bf66fcc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:14.004286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a7df8-a9c3-41ae-b1f7-658d7bf66fcc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:14.005882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a7df8-a9c3-41ae-b1f7-658d7bf66fcc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-896a7df8-a9c3-41ae-b1f7-658d7bf66fcc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:14.015130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:14.016087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:51:14.016998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:14.019792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 449/905] 10.222.0.86 () {56 vars in 1132 bytes} [Wed Aug 24 09:51:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:14.025632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d29000-1bba-49e2-b773-85de4d353c97 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:14.133345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d29000-1bba-49e2-b773-85de4d353c97 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:14.171190 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7618880-afa5-4eae-837c-02f0a31364d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:14.172187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7618880-afa5-4eae-837c-02f0a31364d6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:14.172693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7618880-afa5-4eae-837c-02f0a31364d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:14.173159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7618880-afa5-4eae-837c-02f0a31364d6 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:14.173992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7618880-afa5-4eae-837c-02f0a31364d6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:14.174491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7618880-afa5-4eae-837c-02f0a31364d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:14.176165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7618880-afa5-4eae-837c-02f0a31364d6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:14.182071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 457/906] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Aug 24 09:51:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:14.189139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a7df8-a9c3-41ae-b1f7-658d7bf66fcc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:14.193131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170091-7b81-4558-a515-543c8c10daf7 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:14.193419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170091-7b81-4558-a515-543c8c10daf7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:14.193691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170091-7b81-4558-a515-543c8c10daf7 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:14.195254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 458/907] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:14.200268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6feeb-5a47-4b44-88c0-fcd17d96606c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:14.200549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6feeb-5a47-4b44-88c0-fcd17d96606c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:14.200823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6feeb-5a47-4b44-88c0-fcd17d96606c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:14.215421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:14.219873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-eaa6feeb-5a47-4b44-88c0-fcd17d96606c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:14.221136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-eaa6feeb-5a47-4b44-88c0-fcd17d96606c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:14.296885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:14.310343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 459/908] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:14.320089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a7df8-a9c3-41ae-b1f7-658d7bf66fcc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:14.348772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6db4c-28b5-4609-9c70-f3890f392e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:14.349371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6db4c-28b5-4609-9c70-f3890f392e54 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:14.349652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6db4c-28b5-4609-9c70-f3890f392e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:14.349923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6db4c-28b5-4609-9c70-f3890f392e54 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:14.350550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6db4c-28b5-4609-9c70-f3890f392e54 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:14.350853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6db4c-28b5-4609-9c70-f3890f392e54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:14.352280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6db4c-28b5-4609-9c70-f3890f392e54 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:14.357773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 450/909] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Aug 24 09:51:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:14.364316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6feeb-5a47-4b44-88c0-fcd17d96606c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:14.472792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6feeb-5a47-4b44-88c0-fcd17d96606c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:14.510033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad06440-a5b7-4169-b333-245785e6ffdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:14.511030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad06440-a5b7-4169-b333-245785e6ffdf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:14.511524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad06440-a5b7-4169-b333-245785e6ffdf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:14.511993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad06440-a5b7-4169-b333-245785e6ffdf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:14.512999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad06440-a5b7-4169-b333-245785e6ffdf None admin] RBAC: Adding query filter params (interface=public, service_id=48ca30801bb641fc865ed446bdcc2769, region_id=RegionOne) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:14.513522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad06440-a5b7-4169-b333-245785e6ffdf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:14.515118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad06440-a5b7-4169-b333-245785e6ffdf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:14.521049 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 460/910] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Aug 24 09:51:14 2022] GET /identity/v3/endpoints?service_id=48ca30801bb641fc865ed446bdcc2769&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:14.526853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6db4c-28b5-4609-9c70-f3890f392e54 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:14.571228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad06440-a5b7-4169-b333-245785e6ffdf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:14.628261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:14.635147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:14.648229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6db4c-28b5-4609-9c70-f3890f392e54 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:14.684379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad06440-a5b7-4169-b333-245785e6ffdf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:14.685082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ccfa6-0a39-4fd4-9e01-99edebc18af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:14.685691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ccfa6-0a39-4fd4-9e01-99edebc18af9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:14.685974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ccfa6-0a39-4fd4-9e01-99edebc18af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:14.686250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ccfa6-0a39-4fd4-9e01-99edebc18af9 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:14.686884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ccfa6-0a39-4fd4-9e01-99edebc18af9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:14.688276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ccfa6-0a39-4fd4-9e01-99edebc18af9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:14.694398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 451/911] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Aug 24 09:51:14 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:51:14.695679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9560a6-b843-4da7-818a-42a2309bc070 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:14.695973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9560a6-b843-4da7-818a-42a2309bc070 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:14.696302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9560a6-b843-4da7-818a-42a2309bc070 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:14.697758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 452/912] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:14.703904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643848b1-74c1-4c70-a687-df723bbba390 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:14.704221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643848b1-74c1-4c70-a687-df723bbba390 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:14.704504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643848b1-74c1-4c70-a687-df723bbba390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:14.725205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-643848b1-74c1-4c70-a687-df723bbba390 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:14.726749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-643848b1-74c1-4c70-a687-df723bbba390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:14.735751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38098c-8305-4e9f-9b3b-c64f2dc188bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:14.737415 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38098c-8305-4e9f-9b3b-c64f2dc188bf None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:14.737917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38098c-8305-4e9f-9b3b-c64f2dc188bf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:14.738385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38098c-8305-4e9f-9b3b-c64f2dc188bf None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:14.739716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38098c-8305-4e9f-9b3b-c64f2dc188bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:14.741991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38098c-8305-4e9f-9b3b-c64f2dc188bf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:14.771781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 461/913] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:51:14 2022] POST /identity/v3/services => generated 241 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:51:14.831020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 453/914] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:15.110896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38098c-8305-4e9f-9b3b-c64f2dc188bf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:15.224801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38098c-8305-4e9f-9b3b-c64f2dc188bf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:15.245863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:15.261575 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e5c0b-10e3-4504-828a-6cc81c697a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:15.262204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e5c0b-10e3-4504-828a-6cc81c697a50 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:15.262478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e5c0b-10e3-4504-828a-6cc81c697a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:15.262746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e5c0b-10e3-4504-828a-6cc81c697a50 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:15.263326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e5c0b-10e3-4504-828a-6cc81c697a50 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:15.265401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e5c0b-10e3-4504-828a-6cc81c697a50 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a82e5c0b-10e3-4504-828a-6cc81c697a50 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:15.273951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:15.275043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:15.276542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 462/915] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:51:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:15.280959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643848b1-74c1-4c70-a687-df723bbba390 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:15.359059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:15.386508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643848b1-74c1-4c70-a687-df723bbba390 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:15.427960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca5f52-9787-4282-8819-d322fa94343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:15.428978 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca5f52-9787-4282-8819-d322fa94343e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:15.429467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca5f52-9787-4282-8819-d322fa94343e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:15.429953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca5f52-9787-4282-8819-d322fa94343e None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:15.431049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca5f52-9787-4282-8819-d322fa94343e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:15.431604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca5f52-9787-4282-8819-d322fa94343e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:15.433216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca5f52-9787-4282-8819-d322fa94343e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:15.443259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 454/916] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:51:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:15.445027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e5c0b-10e3-4504-828a-6cc81c697a50 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:15.495490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e244c0-868b-4808-9bfe-13657648e228 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:15.495779 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e244c0-868b-4808-9bfe-13657648e228 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:15.496342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e244c0-868b-4808-9bfe-13657648e228 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:15.498134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 455/917] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:15.504068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a10e-8822-4ef4-afa3-1ab1136956a1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:15.504430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a10e-8822-4ef4-afa3-1ab1136956a1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:15.504780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a10e-8822-4ef4-afa3-1ab1136956a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:15.528029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-76e1a10e-8822-4ef4-afa3-1ab1136956a1 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:15.529434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-76e1a10e-8822-4ef4-afa3-1ab1136956a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:15.554658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e5c0b-10e3-4504-828a-6cc81c697a50 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:15.592887 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab62f12-2df0-43f3-8af5-9b7a84a3310b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:15.593897 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62f12-2df0-43f3-8af5-9b7a84a3310b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:15.594394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62f12-2df0-43f3-8af5-9b7a84a3310b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:15.595048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62f12-2df0-43f3-8af5-9b7a84a3310b None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:15.595891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab62f12-2df0-43f3-8af5-9b7a84a3310b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:15.596578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab62f12-2df0-43f3-8af5-9b7a84a3310b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:15.598174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab62f12-2df0-43f3-8af5-9b7a84a3310b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:15.603850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 463/918] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Aug 24 09:51:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:15.611237 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab62f12-2df0-43f3-8af5-9b7a84a3310b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:15.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 456/919] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:15.669171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:15.726173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab62f12-2df0-43f3-8af5-9b7a84a3310b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:15.761503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0c8ae-0d55-4394-89f3-12293cb15a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:15.762416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0c8ae-0d55-4394-89f3-12293cb15a7a None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:15.762880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0c8ae-0d55-4394-89f3-12293cb15a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:15.763345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0c8ae-0d55-4394-89f3-12293cb15a7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:15.764240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0c8ae-0d55-4394-89f3-12293cb15a7a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:15.766341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0c8ae-0d55-4394-89f3-12293cb15a7a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:15.788460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 464/920] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 09:51:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:51:15.907604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a10e-8822-4ef4-afa3-1ab1136956a1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:15.991377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:16.026434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a10e-8822-4ef4-afa3-1ab1136956a1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:16.067620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07557093-ce85-4bf4-be4b-6be06fe6d8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:16.068275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07557093-ce85-4bf4-be4b-6be06fe6d8eb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:16.068555 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07557093-ce85-4bf4-be4b-6be06fe6d8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:16.068825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07557093-ce85-4bf4-be4b-6be06fe6d8eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:16.079278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07557093-ce85-4bf4-be4b-6be06fe6d8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-07557093-ce85-4bf4-be4b-6be06fe6d8eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:16.083879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:16.084866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:16.086094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 457/921] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Aug 24 09:51:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:16.091406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0c8ae-0d55-4394-89f3-12293cb15a7a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:16.200112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:16.204798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0c8ae-0d55-4394-89f3-12293cb15a7a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:16.247216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24115b86-b940-416b-ae07-6b543b14f2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:16.247748 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24115b86-b940-416b-ae07-6b543b14f2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:16.248029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24115b86-b940-416b-ae07-6b543b14f2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:16.248324 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24115b86-b940-416b-ae07-6b543b14f2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:16.248992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24115b86-b940-416b-ae07-6b543b14f2b1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:16.249324 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24115b86-b940-416b-ae07-6b543b14f2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:16.250794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24115b86-b940-416b-ae07-6b543b14f2b1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:16.264384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 465/922] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Aug 24 09:51:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:16.266721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07557093-ce85-4bf4-be4b-6be06fe6d8eb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:16.374059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07557093-ce85-4bf4-be4b-6be06fe6d8eb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:16.399688 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:16.412463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830635c-170c-4779-aed9-6378db2de255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:16.413089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830635c-170c-4779-aed9-6378db2de255 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:16.413370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830635c-170c-4779-aed9-6378db2de255 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:16.413641 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830635c-170c-4779-aed9-6378db2de255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:16.424251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830635c-170c-4779-aed9-6378db2de255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:16.425706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830635c-170c-4779-aed9-6378db2de255 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7830635c-170c-4779-aed9-6378db2de255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:16.440829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:16.441814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:16.442710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:16.442710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:16.442710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:16.448864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24115b86-b940-416b-ae07-6b543b14f2b1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:16.449876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 458/923] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:16.556433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24115b86-b940-416b-ae07-6b543b14f2b1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:16.592793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04e25c-8fa9-4fb9-b465-58875e029d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:16.593426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04e25c-8fa9-4fb9-b465-58875e029d35 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:16.593706 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04e25c-8fa9-4fb9-b465-58875e029d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:16.593979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04e25c-8fa9-4fb9-b465-58875e029d35 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:16.594621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04e25c-8fa9-4fb9-b465-58875e029d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:16.594920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04e25c-8fa9-4fb9-b465-58875e029d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:16.596320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04e25c-8fa9-4fb9-b465-58875e029d35 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:16.608896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 466/924] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:16.614454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830635c-170c-4779-aed9-6378db2de255 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:16.716385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a9e16-b250-42b9-8dfd-b67d79106c66 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:16.717040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a9e16-b250-42b9-8dfd-b67d79106c66 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:16.717514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a9e16-b250-42b9-8dfd-b67d79106c66 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:16.719624 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830635c-170c-4779-aed9-6378db2de255 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:16.720096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 467/925] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:16.724562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c93af-f367-466c-a769-50975a13e14a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:16.725089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c93af-f367-466c-a769-50975a13e14a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:16.725555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c93af-f367-466c-a769-50975a13e14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:16.745423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5d0c93af-f367-466c-a769-50975a13e14a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:16.747104 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5d0c93af-f367-466c-a769-50975a13e14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:16.760787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3d54d-ef5e-45f8-86a7-fa0fa70c413e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:16.761416 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3d54d-ef5e-45f8-86a7-fa0fa70c413e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:16.761696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3d54d-ef5e-45f8-86a7-fa0fa70c413e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:16.761971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3d54d-ef5e-45f8-86a7-fa0fa70c413e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:16.767443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:16.774345 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3d54d-ef5e-45f8-86a7-fa0fa70c413e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:16.780067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3d54d-ef5e-45f8-86a7-fa0fa70c413e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-62d3d54d-ef5e-45f8-86a7-fa0fa70c413e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:16.796088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:16.797652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 459/926] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Aug 24 09:51:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:16.798706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3d54d-ef5e-45f8-86a7-fa0fa70c413e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:16.835122 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:16.841756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 468/927] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:16.849607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da4cbc-8d67-4c57-b8d7-53ac113c0262 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:16.850016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da4cbc-8d67-4c57-b8d7-53ac113c0262 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:16.850416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da4cbc-8d67-4c57-b8d7-53ac113c0262 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:16.852070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 469/928] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:16.856863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0940f-3269-4f17-9a40-b65af959e05b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:16.857251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0940f-3269-4f17-9a40-b65af959e05b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:16.857627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0940f-3269-4f17-9a40-b65af959e05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:16.877014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2ce0940f-3269-4f17-9a40-b65af959e05b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:16.878353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2ce0940f-3269-4f17-9a40-b65af959e05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:16.907261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3d54d-ef5e-45f8-86a7-fa0fa70c413e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:16.943149 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c455c4-bb80-443b-a567-4c1954c85f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:16.943778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c455c4-bb80-443b-a567-4c1954c85f60 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:16.944066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c455c4-bb80-443b-a567-4c1954c85f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:16.944376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c455c4-bb80-443b-a567-4c1954c85f60 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:16.945234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c455c4-bb80-443b-a567-4c1954c85f60 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:16.945540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c455c4-bb80-443b-a567-4c1954c85f60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:16.946982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c455c4-bb80-443b-a567-4c1954c85f60 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:16.960456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 470/929] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:16.969616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 460/930] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Aug 24 09:51:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:16.970909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0940f-3269-4f17-9a40-b65af959e05b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.058561 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:17.078649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0940f-3269-4f17-9a40-b65af959e05b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:17.109134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c455c4-bb80-443b-a567-4c1954c85f60 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.114028 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116db64-1b1e-4fd6-9144-23377d6769da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:17.114660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116db64-1b1e-4fd6-9144-23377d6769da None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:17.114936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116db64-1b1e-4fd6-9144-23377d6769da None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:17.115207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116db64-1b1e-4fd6-9144-23377d6769da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:17.126361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116db64-1b1e-4fd6-9144-23377d6769da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:17.127821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116db64-1b1e-4fd6-9144-23377d6769da None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-b116db64-1b1e-4fd6-9144-23377d6769da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:17.142266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:17.143209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:17.144151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:17.144151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:17.144151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:17.144151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 471/931] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:17.150860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116db64-1b1e-4fd6-9144-23377d6769da None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.219796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c455c4-bb80-443b-a567-4c1954c85f60 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:17.237896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:17.267141 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06914a-e5f7-4c68-8c58-dfdabe7fb3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:17.267765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06914a-e5f7-4c68-8c58-dfdabe7fb3fa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:17.268064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06914a-e5f7-4c68-8c58-dfdabe7fb3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:17.268342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06914a-e5f7-4c68-8c58-dfdabe7fb3fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:17.268925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06914a-e5f7-4c68-8c58-dfdabe7fb3fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:17.269847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116db64-1b1e-4fd6-9144-23377d6769da None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:17.270294 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06914a-e5f7-4c68-8c58-dfdabe7fb3fa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ae06914a-e5f7-4c68-8c58-dfdabe7fb3fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:17.279003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:17.280235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:17.281486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 461/932] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Aug 24 09:51:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:17.282558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06914a-e5f7-4c68-8c58-dfdabe7fb3fa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.315924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1b965-7433-4f33-b575-953bd48007db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:17.316565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1b965-7433-4f33-b575-953bd48007db None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:17.316841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1b965-7433-4f33-b575-953bd48007db None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:17.317107 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1b965-7433-4f33-b575-953bd48007db None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:17.317863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1b965-7433-4f33-b575-953bd48007db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:17.318275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1b965-7433-4f33-b575-953bd48007db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:17.319647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1b965-7433-4f33-b575-953bd48007db None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:17.321540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:17.339626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 472/933] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:17.340401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1b965-7433-4f33-b575-953bd48007db None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.389661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06914a-e5f7-4c68-8c58-dfdabe7fb3fa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:17.427090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56588f60-33ae-4a2c-99cb-830429c82a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:17.427761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56588f60-33ae-4a2c-99cb-830429c82a2e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:17.428066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56588f60-33ae-4a2c-99cb-830429c82a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:17.428353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56588f60-33ae-4a2c-99cb-830429c82a2e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:17.428934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56588f60-33ae-4a2c-99cb-830429c82a2e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:17.430300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56588f60-33ae-4a2c-99cb-830429c82a2e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-56588f60-33ae-4a2c-99cb-830429c82a2e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:17.440065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:17.441047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:17.441955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 462/934] 10.222.0.86 () {56 vars in 1132 bytes} [Wed Aug 24 09:51:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:17.442744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56588f60-33ae-4a2c-99cb-830429c82a2e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.451039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1b965-7433-4f33-b575-953bd48007db None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:17.472304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:17.489669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0416f7b-925f-4731-94d4-d2b2e22506df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:17.490262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0416f7b-925f-4731-94d4-d2b2e22506df None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:17.490543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0416f7b-925f-4731-94d4-d2b2e22506df None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:17.490814 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0416f7b-925f-4731-94d4-d2b2e22506df None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:17.491437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0416f7b-925f-4731-94d4-d2b2e22506df None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:17.491742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0416f7b-925f-4731-94d4-d2b2e22506df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:17.493162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0416f7b-925f-4731-94d4-d2b2e22506df None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:17.503130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 473/935] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Aug 24 09:51:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:17.503130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0416f7b-925f-4731-94d4-d2b2e22506df None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.559116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56588f60-33ae-4a2c-99cb-830429c82a2e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:17.564323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:17.595050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93530934-2479-4947-846c-58019d2ca8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:17.595679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93530934-2479-4947-846c-58019d2ca8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:17.595963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93530934-2479-4947-846c-58019d2ca8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:17.596268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93530934-2479-4947-846c-58019d2ca8e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:17.602980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93530934-2479-4947-846c-58019d2ca8e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:17.604453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93530934-2479-4947-846c-58019d2ca8e1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:17.613904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0416f7b-925f-4731-94d4-d2b2e22506df None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-93530934-2479-4947-846c-58019d2ca8e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:17.624088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:17.625434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:17.626389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:17.626389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:17.626389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:17.626389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:17.626389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:17.626389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:17.626389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:17.626389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:17.630165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 463/936] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:17.631040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93530934-2479-4947-846c-58019d2ca8e1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.656415 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806668bb-d46b-4311-9d83-bb470f952210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:17.657511 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806668bb-d46b-4311-9d83-bb470f952210 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:17.658220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806668bb-d46b-4311-9d83-bb470f952210 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:17.658709 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806668bb-d46b-4311-9d83-bb470f952210 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:17.659556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806668bb-d46b-4311-9d83-bb470f952210 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:17.660198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806668bb-d46b-4311-9d83-bb470f952210 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:17.661918 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806668bb-d46b-4311-9d83-bb470f952210 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:17.667353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 474/937] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Aug 24 09:51:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:17.668438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806668bb-d46b-4311-9d83-bb470f952210 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.726385 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:17.749392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93530934-2479-4947-846c-58019d2ca8e1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:17.791420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b25c5-8389-4828-b1b7-b219457ee60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:17.792024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b25c5-8389-4828-b1b7-b219457ee60e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:17.792323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b25c5-8389-4828-b1b7-b219457ee60e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:17.792643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b25c5-8389-4828-b1b7-b219457ee60e None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:17.793216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b25c5-8389-4828-b1b7-b219457ee60e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:17.793515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b25c5-8389-4828-b1b7-b219457ee60e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:17.794756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806668bb-d46b-4311-9d83-bb470f952210 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:17.794939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b25c5-8389-4828-b1b7-b219457ee60e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:17.805180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 464/938] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Aug 24 09:51:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:17.806314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b25c5-8389-4828-b1b7-b219457ee60e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.819517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:17.835600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016baa29-6296-4fb5-a3e6-de4af4a598ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:17.836244 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016baa29-6296-4fb5-a3e6-de4af4a598ab None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:17.836520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016baa29-6296-4fb5-a3e6-de4af4a598ab None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:17.836789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016baa29-6296-4fb5-a3e6-de4af4a598ab None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:17.837475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016baa29-6296-4fb5-a3e6-de4af4a598ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:17.837779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016baa29-6296-4fb5-a3e6-de4af4a598ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:17.839243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016baa29-6296-4fb5-a3e6-de4af4a598ab None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:17.851735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 475/939] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:51:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:17.852404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016baa29-6296-4fb5-a3e6-de4af4a598ab None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.920142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b25c5-8389-4828-b1b7-b219457ee60e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:17.953866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a9cd5-7c1a-494d-86ce-b064e0b600ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:17.954551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a9cd5-7c1a-494d-86ce-b064e0b600ca None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:17.954835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a9cd5-7c1a-494d-86ce-b064e0b600ca None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:17.955110 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a9cd5-7c1a-494d-86ce-b064e0b600ca None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:17.955738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a9cd5-7c1a-494d-86ce-b064e0b600ca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:17.956071 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a9cd5-7c1a-494d-86ce-b064e0b600ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:17.957470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a9cd5-7c1a-494d-86ce-b064e0b600ca None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:17.957828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016baa29-6296-4fb5-a3e6-de4af4a598ab None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:17.963228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 465/940] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Aug 24 09:51:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:17.964001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a9cd5-7c1a-494d-86ce-b064e0b600ca None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:17.979604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:17.999019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5ab29-0ee2-4fe7-abb5-c62f893d0c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:17.999676 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5ab29-0ee2-4fe7-abb5-c62f893d0c9f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:17.999959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5ab29-0ee2-4fe7-abb5-c62f893d0c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:18.000262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5ab29-0ee2-4fe7-abb5-c62f893d0c9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:18.000962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5ab29-0ee2-4fe7-abb5-c62f893d0c9f None admin] RBAC: Adding query filter params (interface=public, service_id=88b28e796f7e494db9d691ef3c8182f4, region_id=RegionOne) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:18.001268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5ab29-0ee2-4fe7-abb5-c62f893d0c9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:18.002633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5ab29-0ee2-4fe7-abb5-c62f893d0c9f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:18.008461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 476/941] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Aug 24 09:51:17 2022] GET /identity/v3/endpoints?service_id=88b28e796f7e494db9d691ef3c8182f4&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:18.009614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf42f9-7099-4afe-81b9-eed295c4564e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:18.009895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf42f9-7099-4afe-81b9-eed295c4564e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:18.010169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf42f9-7099-4afe-81b9-eed295c4564e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:18.011356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 477/942] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:18.012033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf42f9-7099-4afe-81b9-eed295c4564e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:18.059623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:18.076237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a9cd5-7c1a-494d-86ce-b064e0b600ca None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:18.109339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4d1bd-4765-467c-8696-80feb1147748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:18.110071 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4d1bd-4765-467c-8696-80feb1147748 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:18.110357 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4d1bd-4765-467c-8696-80feb1147748 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:18.110635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4d1bd-4765-467c-8696-80feb1147748 None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/1bac9dc42db9451e9e87a88c17f29c80/roles/641561cf524144e5a49f595528477389` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:18.112962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4d1bd-4765-467c-8696-80feb1147748 None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=1bac9dc42db9451e9e87a88c17f29c80, role_id=641561cf524144e5a49f595528477389)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:18.114531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4d1bd-4765-467c-8696-80feb1147748 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:18.114821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf42f9-7099-4afe-81b9-eed295c4564e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:18.131321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 466/943] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:51:17 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/1bac9dc42db9451e9e87a88c17f29c80/roles/641561cf524144e5a49f595528477389 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:51:18.132176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4d1bd-4765-467c-8696-80feb1147748 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:18.162210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7aacd-e852-4529-9307-56c2c8ebeed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:18.163236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7aacd-e852-4529-9307-56c2c8ebeed4 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:18.163519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7aacd-e852-4529-9307-56c2c8ebeed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:18.163792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7aacd-e852-4529-9307-56c2c8ebeed4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:18.164519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7aacd-e852-4529-9307-56c2c8ebeed4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:18.165958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7aacd-e852-4529-9307-56c2c8ebeed4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:18.191374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 478/944] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 09:51:18 2022] POST /identity/v3/endpoints => generated 347 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:51:18.192878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281433-eedb-402d-9c7c-c2b108adeabb None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:18.193222 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281433-eedb-402d-9c7c-c2b108adeabb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:18.193558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281433-eedb-402d-9c7c-c2b108adeabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:18.213760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6f281433-eedb-402d-9c7c-c2b108adeabb None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:18.215150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6f281433-eedb-402d-9c7c-c2b108adeabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:18.252578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:18.297490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4d1bd-4765-467c-8696-80feb1147748 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:18.318412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 479/945] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:18.343447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f092e7-facd-4e23-9160-f088b4352110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:18.344211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f092e7-facd-4e23-9160-f088b4352110 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:18.344423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f092e7-facd-4e23-9160-f088b4352110 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:18.344707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f092e7-facd-4e23-9160-f088b4352110 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:18.345328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f092e7-facd-4e23-9160-f088b4352110 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:18.346718 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f092e7-facd-4e23-9160-f088b4352110 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:18.357148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 467/946] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Aug 24 09:51:18 2022] GET /identity/v3/services => generated 1497 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:51:18.611679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f281433-eedb-402d-9c7c-c2b108adeabb None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:18.641720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:18.720822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f281433-eedb-402d-9c7c-c2b108adeabb None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:18.755619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f14dd5-3abe-4eae-bbd7-d2bbcc36d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:18.756257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f14dd5-3abe-4eae-bbd7-d2bbcc36d57c None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:18.756538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f14dd5-3abe-4eae-bbd7-d2bbcc36d57c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:18.756807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f14dd5-3abe-4eae-bbd7-d2bbcc36d57c None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:18.757454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f14dd5-3abe-4eae-bbd7-d2bbcc36d57c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:18.758902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f14dd5-3abe-4eae-bbd7-d2bbcc36d57c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:18.761542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: WARNING keystone.server.flask.common [None req-24f14dd5-3abe-4eae-bbd7-d2bbcc36d57c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 09:51:18.780605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.sql.core [None req-24f14dd5-3abe-4eae-bbd7-d2bbcc36d57c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:51:18.780605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 09:51:18.780605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [parameters: {'id': 'dd3fb7be65c34e60b742dfe52cdd7b6e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 09:51:18.780605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81464) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-24f14dd5-3abe-4eae-bbd7-d2bbcc36d57c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dd3fb7be65c34e60b742dfe52cdd7b6e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dd3fb7be65c34e60b742dfe52cdd7b6e. Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:51:18.798157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application result.read() Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 09:51:18.799040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application yield resource Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise self.value Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application session.commit() Aug 24 09:51:18.799972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:51:18.800892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:51:18.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application result.read() Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:51:18.802622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [parameters: {'id': 'dd3fb7be65c34e60b742dfe52cdd7b6e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:18.803478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:18.805749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:18.808241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:18.808241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 09:51:18.808241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 09:51:18.808241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:18.808241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:18.808241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 09:51:18.808241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 09:51:18.808241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dd3fb7be65c34e60b742dfe52cdd7b6e. Aug 24 09:51:18.808241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:18.808241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 480/947] 10.222.0.86 () {60 vars in 1140 bytes} [Wed Aug 24 09:51:18 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:18.809517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f092e7-facd-4e23-9160-f088b4352110 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:18.926165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f092e7-facd-4e23-9160-f088b4352110 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:18.962155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2d18a-054e-43cf-b4aa-ffe753380ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:18.962785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2d18a-054e-43cf-b4aa-ffe753380ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:18.963066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2d18a-054e-43cf-b4aa-ffe753380ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:18.963344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2d18a-054e-43cf-b4aa-ffe753380ff6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:18.965367 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:18.970492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2d18a-054e-43cf-b4aa-ffe753380ff6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:18.972002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2d18a-054e-43cf-b4aa-ffe753380ff6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-16c2d18a-054e-43cf-b4aa-ffe753380ff6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:18.983563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:18.984517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:18.985420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:18.985420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:18.985420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:18.985420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:18.985420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:18.985420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:18.985420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:18.985420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:18.988277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 468/948] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:18.993363 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f14dd5-3abe-4eae-bbd7-d2bbcc36d57c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:19.085158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:19.102880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f14dd5-3abe-4eae-bbd7-d2bbcc36d57c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:19.137402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c6bb0-d54b-43a8-829d-1f1fea7100fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:19.138007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c6bb0-d54b-43a8-829d-1f1fea7100fb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:19.138288 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c6bb0-d54b-43a8-829d-1f1fea7100fb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:19.138561 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c6bb0-d54b-43a8-829d-1f1fea7100fb None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:19.139228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c6bb0-d54b-43a8-829d-1f1fea7100fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:19.139533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c6bb0-d54b-43a8-829d-1f1fea7100fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:19.141774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c6bb0-d54b-43a8-829d-1f1fea7100fb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:19.164435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 481/949] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:51:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:20.174976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4c12c-0b93-4b5e-bec9-9891341c44e4 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:20.175476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4c12c-0b93-4b5e-bec9-9891341c44e4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:20.175476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4c12c-0b93-4b5e-bec9-9891341c44e4 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:20.177099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 469/950] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:20.183040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1ce42-e8cc-4d06-a38f-ea5aa671a9f0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:20.183917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1ce42-e8cc-4d06-a38f-ea5aa671a9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:20.184424 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1ce42-e8cc-4d06-a38f-ea5aa671a9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:20.203931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-69b1ce42-e8cc-4d06-a38f-ea5aa671a9f0 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:20.205811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-69b1ce42-e8cc-4d06-a38f-ea5aa671a9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:20.241724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0fe62-bcc7-41cd-9473-a2b17f44e403 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:20.242063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0fe62-bcc7-41cd-9473-a2b17f44e403 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:20.242281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0fe62-bcc7-41cd-9473-a2b17f44e403 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:20.244013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 470/951] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:20.249093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6ac1-654d-4c7d-bdf3-7c2589884fa5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:20.249374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6ac1-654d-4c7d-bdf3-7c2589884fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:20.249791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6ac1-654d-4c7d-bdf3-7c2589884fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:20.270045 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c32c6ac1-654d-4c7d-bdf3-7c2589884fa5 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:20.271491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c32c6ac1-654d-4c7d-bdf3-7c2589884fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:20.292132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 482/952] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:20.323329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b8df4-923f-411c-9d00-e970f19149de None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:20.323666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b8df4-923f-411c-9d00-e970f19149de None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:20.323879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b8df4-923f-411c-9d00-e970f19149de None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:20.325621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 483/953] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:20.334062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143b23-6740-4c51-ba59-b1a8f53add73 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:20.334350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143b23-6740-4c51-ba59-b1a8f53add73 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:20.334621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143b23-6740-4c51-ba59-b1a8f53add73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:20.352875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-50143b23-6740-4c51-ba59-b1a8f53add73 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:20.354050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-50143b23-6740-4c51-ba59-b1a8f53add73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:20.360994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 471/954] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:20.434937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 484/955] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:20.572055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c6ac1-654d-4c7d-bdf3-7c2589884fa5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:20.637807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50143b23-6740-4c51-ba59-b1a8f53add73 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:20.642450 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:20.674274 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c6ac1-654d-4c7d-bdf3-7c2589884fa5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:20.699244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:20.710036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd666d-8674-4938-bb73-eb12bc983f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:20.710659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd666d-8674-4938-bb73-eb12bc983f46 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:20.710933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd666d-8674-4938-bb73-eb12bc983f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:20.711201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd666d-8674-4938-bb73-eb12bc983f46 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:20.721167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd666d-8674-4938-bb73-eb12bc983f46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-cefd666d-8674-4938-bb73-eb12bc983f46 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:20.725731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:20.727012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 472/956] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Aug 24 09:51:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:20.728722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd666d-8674-4938-bb73-eb12bc983f46 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:20.759239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50143b23-6740-4c51-ba59-b1a8f53add73 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:20.799409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576bfe2-26e5-4e2e-bc13-0f4ebe900f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:20.800175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576bfe2-26e5-4e2e-bc13-0f4ebe900f78 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:20.800524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576bfe2-26e5-4e2e-bc13-0f4ebe900f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:20.800803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576bfe2-26e5-4e2e-bc13-0f4ebe900f78 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:20.801444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576bfe2-26e5-4e2e-bc13-0f4ebe900f78 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:20.802827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576bfe2-26e5-4e2e-bc13-0f4ebe900f78 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-a576bfe2-26e5-4e2e-bc13-0f4ebe900f78 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:20.812331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:20.814059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:20.815007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 485/957] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:20.815822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576bfe2-26e5-4e2e-bc13-0f4ebe900f78 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:20.854863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd666d-8674-4938-bb73-eb12bc983f46 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:20.888704 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:20.892654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20839db1-e52e-4c0c-8286-8b4489db881a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:20.893291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20839db1-e52e-4c0c-8286-8b4489db881a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:20.893620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20839db1-e52e-4c0c-8286-8b4489db881a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:20.893904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20839db1-e52e-4c0c-8286-8b4489db881a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:20.894500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20839db1-e52e-4c0c-8286-8b4489db881a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:20.895871 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20839db1-e52e-4c0c-8286-8b4489db881a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-20839db1-e52e-4c0c-8286-8b4489db881a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:20.905024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:20.906112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:20.906982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 473/958] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Aug 24 09:51:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:20.908489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20839db1-e52e-4c0c-8286-8b4489db881a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:20.930830 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576bfe2-26e5-4e2e-bc13-0f4ebe900f78 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:20.959267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:20.967770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d00b04-873a-45aa-929d-3a97fb1e38cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:20.968403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d00b04-873a-45aa-929d-3a97fb1e38cf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:20.968712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d00b04-873a-45aa-929d-3a97fb1e38cf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:20.968981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d00b04-873a-45aa-929d-3a97fb1e38cf None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:20.969650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d00b04-873a-45aa-929d-3a97fb1e38cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:20.969953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d00b04-873a-45aa-929d-3a97fb1e38cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:20.971339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d00b04-873a-45aa-929d-3a97fb1e38cf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:20.980076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 486/959] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Aug 24 09:51:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:20.986019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d00b04-873a-45aa-929d-3a97fb1e38cf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:21.016251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20839db1-e52e-4c0c-8286-8b4489db881a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:21.053205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b513bb-d0f5-4fcb-9077-96d533fc6548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:21.054166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b513bb-d0f5-4fcb-9077-96d533fc6548 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:21.054672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b513bb-d0f5-4fcb-9077-96d533fc6548 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:21.055181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b513bb-d0f5-4fcb-9077-96d533fc6548 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:21.055888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b513bb-d0f5-4fcb-9077-96d533fc6548 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:21.056227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b513bb-d0f5-4fcb-9077-96d533fc6548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:21.057850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b513bb-d0f5-4fcb-9077-96d533fc6548 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:21.063757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 474/960] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:51:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:21.069226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b513bb-d0f5-4fcb-9077-96d533fc6548 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:21.095717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d00b04-873a-45aa-929d-3a97fb1e38cf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:21.129144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:21.136158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d87793-8b35-4ddc-a563-a5db8fae281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:21.136764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d87793-8b35-4ddc-a563-a5db8fae281d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:21.137039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d87793-8b35-4ddc-a563-a5db8fae281d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:21.137309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d87793-8b35-4ddc-a563-a5db8fae281d None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:21.137994 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d87793-8b35-4ddc-a563-a5db8fae281d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:21.138295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d87793-8b35-4ddc-a563-a5db8fae281d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:21.139682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d87793-8b35-4ddc-a563-a5db8fae281d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:21.145343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 487/961] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Aug 24 09:51:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:21.145937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d87793-8b35-4ddc-a563-a5db8fae281d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:21.182735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b513bb-d0f5-4fcb-9077-96d533fc6548 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:21.195879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:21.218704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f2ec-3057-4e13-81f8-55941fecb733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:21.219257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f2ec-3057-4e13-81f8-55941fecb733 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:21.219538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f2ec-3057-4e13-81f8-55941fecb733 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:21.219815 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f2ec-3057-4e13-81f8-55941fecb733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:21.230750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64f2ec-3057-4e13-81f8-55941fecb733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:21.232219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64f2ec-3057-4e13-81f8-55941fecb733 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4e64f2ec-3057-4e13-81f8-55941fecb733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:21.246508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:21.247531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:21.248491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:21.248491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:21.248491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:21.248673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d87793-8b35-4ddc-a563-a5db8fae281d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:21.249077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 475/962] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:21.249077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f2ec-3057-4e13-81f8-55941fecb733 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:21.287252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bb326-8aa2-437c-94fe-a93890752974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:21.287857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bb326-8aa2-437c-94fe-a93890752974 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:21.288172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bb326-8aa2-437c-94fe-a93890752974 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:21.288451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bb326-8aa2-437c-94fe-a93890752974 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:21.289330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686bb326-8aa2-437c-94fe-a93890752974 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:21.289466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686bb326-8aa2-437c-94fe-a93890752974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:21.290870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686bb326-8aa2-437c-94fe-a93890752974 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:21.308092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 488/963] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:51:21 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:21.308092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fe80f-48b3-4eed-8d3d-88b4511ae387 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:21.308092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fe80f-48b3-4eed-8d3d-88b4511ae387 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:21.308092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fe80f-48b3-4eed-8d3d-88b4511ae387 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:21.308092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 489/964] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:21.308092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fe80f-48b3-4eed-8d3d-88b4511ae387 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:21.359438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f2ec-3057-4e13-81f8-55941fecb733 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:21.371184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:21.401724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412ce77-8055-4a7e-9e91-d2cc0be5859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:21.402347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412ce77-8055-4a7e-9e91-d2cc0be5859a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:21.402651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412ce77-8055-4a7e-9e91-d2cc0be5859a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:21.402938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412ce77-8055-4a7e-9e91-d2cc0be5859a None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:21.403780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412ce77-8055-4a7e-9e91-d2cc0be5859a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:21.404338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412ce77-8055-4a7e-9e91-d2cc0be5859a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:21.407007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412ce77-8055-4a7e-9e91-d2cc0be5859a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:21.415574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 476/965] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Aug 24 09:51:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:21.415574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a97181-baac-4974-b501-032a45102433 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:21.415574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a97181-baac-4974-b501-032a45102433 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:21.415574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a97181-baac-4974-b501-032a45102433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:21.434935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fe80f-48b3-4eed-8d3d-88b4511ae387 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:21.438860 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-32a97181-baac-4974-b501-032a45102433 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:21.440437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-32a97181-baac-4974-b501-032a45102433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:21.467924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:21.481708 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435144f-c395-4719-915c-ede1a949c5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:21.482243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435144f-c395-4719-915c-ede1a949c5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:21.482523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435144f-c395-4719-915c-ede1a949c5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:21.482795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435144f-c395-4719-915c-ede1a949c5e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:21.483470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435144f-c395-4719-915c-ede1a949c5e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:21.483774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435144f-c395-4719-915c-ede1a949c5e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:21.485804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435144f-c395-4719-915c-ede1a949c5e1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:21.498099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 490/966] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:21.498899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435144f-c395-4719-915c-ede1a949c5e1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:21.528113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 477/967] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:21.528113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a97181-baac-4974-b501-032a45102433 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:21.623287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435144f-c395-4719-915c-ede1a949c5e1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:21.643533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:21.664220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a97181-baac-4974-b501-032a45102433 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:21.665718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b6207-4c73-42b1-acb8-691e36abf5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:21.666302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b6207-4c73-42b1-acb8-691e36abf5a8 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:21.666578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b6207-4c73-42b1-acb8-691e36abf5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:21.666848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b6207-4c73-42b1-acb8-691e36abf5a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:21.667506 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b6207-4c73-42b1-acb8-691e36abf5a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:21.668916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b6207-4c73-42b1-acb8-691e36abf5a8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:21.691779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 491/968] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 09:51:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:51:21.718520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52b20c-8a7c-4ba7-aa58-a22e4d138448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:21.719218 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52b20c-8a7c-4ba7-aa58-a22e4d138448 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:21.719551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52b20c-8a7c-4ba7-aa58-a22e4d138448 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:21.719827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52b20c-8a7c-4ba7-aa58-a22e4d138448 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:21.728640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52b20c-8a7c-4ba7-aa58-a22e4d138448 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:21.730355 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52b20c-8a7c-4ba7-aa58-a22e4d138448 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-da52b20c-8a7c-4ba7-aa58-a22e4d138448 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:21.745171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:21.746167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:21.747128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:21.747128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:21.747128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:21.747128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:21.747128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:21.747128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:21.747128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:21.747128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:21.747128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:51:21.747128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:21.747128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 478/969] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Aug 24 09:51:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:21.753136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b6207-4c73-42b1-acb8-691e36abf5a8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:21.828718 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52b20c-8a7c-4ba7-aa58-a22e4d138448 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:21.852318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:21.872850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b6207-4c73-42b1-acb8-691e36abf5a8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:21.914541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6352c5-30c3-4ae6-bc4c-124be2886f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:21.914925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6352c5-30c3-4ae6-bc4c-124be2886f5f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:21.915175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6352c5-30c3-4ae6-bc4c-124be2886f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:21.915452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6352c5-30c3-4ae6-bc4c-124be2886f5f None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:21.916214 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6352c5-30c3-4ae6-bc4c-124be2886f5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:21.916523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6352c5-30c3-4ae6-bc4c-124be2886f5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:21.917963 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6352c5-30c3-4ae6-bc4c-124be2886f5f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:21.944078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 492/970] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Aug 24 09:51:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:21.946674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6352c5-30c3-4ae6-bc4c-124be2886f5f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:21.967353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52b20c-8a7c-4ba7-aa58-a22e4d138448 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:21.988417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:22.006673 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc7261-fb28-4ffe-aa32-37143d783f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:22.007366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc7261-fb28-4ffe-aa32-37143d783f87 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:22.007579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc7261-fb28-4ffe-aa32-37143d783f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:22.007757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc7261-fb28-4ffe-aa32-37143d783f87 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:22.008788 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc7261-fb28-4ffe-aa32-37143d783f87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:22.010716 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc7261-fb28-4ffe-aa32-37143d783f87 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:22.013302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: WARNING keystone.server.flask.common [None req-e4cc7261-fb28-4ffe-aa32-37143d783f87 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 09:51:22.026352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.sql.core [None req-e4cc7261-fb28-4ffe-aa32-37143d783f87 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:51:22.026352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 09:51:22.026352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [parameters: {'user_id': '157f90d73c214078b12d59a4cb7f76d5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 09:51:22.026352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81463) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e4cc7261-fb28-4ffe-aa32-37143d783f87 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Aug 24 09:51:22.036839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:51:22.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application yield resource Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise self.value Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application session.commit() Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:51:22.039113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:51:22.040271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:51:22.041180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:51:22.042040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [parameters: {'user_id': '157f90d73c214078b12d59a4cb7f76d5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:22.042899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 09:51:22.045604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 09:51:22.048682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 09:51:22.048682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 09:51:22.048682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:51:22.048682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:51:22.048682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 09:51:22.048682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:22.048682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 479/971] 10.222.0.86 () {60 vars in 1131 bytes} [Wed Aug 24 09:51:21 2022] POST /identity/v3/users => generated 160 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:22.048682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc7261-fb28-4ffe-aa32-37143d783f87 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:22.095148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6352c5-30c3-4ae6-bc4c-124be2886f5f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:22.139971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05e3bf-5c40-4abe-8cd1-544de20e7ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:22.141541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05e3bf-5c40-4abe-8cd1-544de20e7ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:22.142030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05e3bf-5c40-4abe-8cd1-544de20e7ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:22.142510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05e3bf-5c40-4abe-8cd1-544de20e7ae8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:22.148339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:22.183561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc7261-fb28-4ffe-aa32-37143d783f87 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:22.229833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5fcaf-737f-4d72-badb-6bbf96354efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:22.230664 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5fcaf-737f-4d72-badb-6bbf96354efa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:22.231061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5fcaf-737f-4d72-badb-6bbf96354efa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:22.231343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5fcaf-737f-4d72-badb-6bbf96354efa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:22.232245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05e3bf-5c40-4abe-8cd1-544de20e7ae8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:22.233922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05e3bf-5c40-4abe-8cd1-544de20e7ae8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:22.241363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5fcaf-737f-4d72-badb-6bbf96354efa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:22.243064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5fcaf-737f-4d72-badb-6bbf96354efa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3cb5fcaf-737f-4d72-badb-6bbf96354efa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:22.259279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:22.260277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:22.261270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:22.261270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:22.261270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:22.261270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:22.261270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:22.261270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:22.261270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:22.261270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:22.261270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:51:22.261270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:22.264193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 480/972] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:51:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:22.266152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5fcaf-737f-4d72-badb-6bbf96354efa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:22.278892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:22.378157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 493/973] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:21 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:51:22.378959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5fcaf-737f-4d72-badb-6bbf96354efa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:22.383647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05e3bf-5c40-4abe-8cd1-544de20e7ae8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:22.419707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acef69-c306-4193-afc3-06080aab3ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:22.420777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acef69-c306-4193-afc3-06080aab3ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:22.421264 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acef69-c306-4193-afc3-06080aab3ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:22.421812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acef69-c306-4193-afc3-06080aab3ff9 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:22.423403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acef69-c306-4193-afc3-06080aab3ff9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:22.423924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acef69-c306-4193-afc3-06080aab3ff9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:22.425602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acef69-c306-4193-afc3-06080aab3ff9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:22.437352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:22.469924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 481/974] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Aug 24 09:51:22 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:22.496578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05e3bf-5c40-4abe-8cd1-544de20e7ae8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:22.531605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:22.538138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55da8a-41c9-48f2-a196-c40d59ebf98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:22.539122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55da8a-41c9-48f2-a196-c40d59ebf98a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:22.539605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55da8a-41c9-48f2-a196-c40d59ebf98a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:22.540132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55da8a-41c9-48f2-a196-c40d59ebf98a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:22.552741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55da8a-41c9-48f2-a196-c40d59ebf98a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:22.554547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55da8a-41c9-48f2-a196-c40d59ebf98a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-aa55da8a-41c9-48f2-a196-c40d59ebf98a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:22.571698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:22.572766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:22.573759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:22.573759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:22.573759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:22.575961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 494/975] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:22.582655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acef69-c306-4193-afc3-06080aab3ff9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:22.693674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acef69-c306-4193-afc3-06080aab3ff9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:22.734178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebc6b5-4a08-43ca-956a-31315b9b7c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:22.735343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebc6b5-4a08-43ca-956a-31315b9b7c6b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:22.735616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebc6b5-4a08-43ca-956a-31315b9b7c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:22.735882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebc6b5-4a08-43ca-956a-31315b9b7c6b None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:22.736559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebc6b5-4a08-43ca-956a-31315b9b7c6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:22.736861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebc6b5-4a08-43ca-956a-31315b9b7c6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:22.738235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebc6b5-4a08-43ca-956a-31315b9b7c6b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:22.751624 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 482/976] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:22.757035 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55da8a-41c9-48f2-a196-c40d59ebf98a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:22.866178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55da8a-41c9-48f2-a196-c40d59ebf98a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:22.900425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b840a0-8956-47a4-aa0b-34ac6ffb5a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:22.901045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b840a0-8956-47a4-aa0b-34ac6ffb5a69 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:22.901325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b840a0-8956-47a4-aa0b-34ac6ffb5a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:22.901600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b840a0-8956-47a4-aa0b-34ac6ffb5a69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:22.908423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b840a0-8956-47a4-aa0b-34ac6ffb5a69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:22.909869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b840a0-8956-47a4-aa0b-34ac6ffb5a69 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-71b840a0-8956-47a4-aa0b-34ac6ffb5a69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:22.921004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:22.921920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:22.928165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:22.928165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:22.928165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:22.928165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:22.928165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:22.928165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:22.928165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:22.928165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:22.928165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 495/977] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:22.930418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebc6b5-4a08-43ca-956a-31315b9b7c6b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:22.990038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:23.037221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebc6b5-4a08-43ca-956a-31315b9b7c6b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:23.072453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe219d2d-348e-4731-8d16-faf8ec234d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:23.073056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219d2d-348e-4731-8d16-faf8ec234d78 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:23.073338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219d2d-348e-4731-8d16-faf8ec234d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:23.073611 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219d2d-348e-4731-8d16-faf8ec234d78 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:23.074298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe219d2d-348e-4731-8d16-faf8ec234d78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:23.074611 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe219d2d-348e-4731-8d16-faf8ec234d78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:23.076089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe219d2d-348e-4731-8d16-faf8ec234d78 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:23.095206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 483/978] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:23.096569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b840a0-8956-47a4-aa0b-34ac6ffb5a69 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:23.145456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:23.202683 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b840a0-8956-47a4-aa0b-34ac6ffb5a69 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:23.236865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553278f-55d2-47ba-9e83-19f7d6bf875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:23.237471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553278f-55d2-47ba-9e83-19f7d6bf875e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:23.237754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553278f-55d2-47ba-9e83-19f7d6bf875e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:23.238030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553278f-55d2-47ba-9e83-19f7d6bf875e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:23.238763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553278f-55d2-47ba-9e83-19f7d6bf875e None admin] RBAC: Adding query filter params (role.id=641561cf524144e5a49f595528477389, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=1bac9dc42db9451e9e87a88c17f29c80) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:23.239078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553278f-55d2-47ba-9e83-19f7d6bf875e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:23.240572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553278f-55d2-47ba-9e83-19f7d6bf875e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:23.252082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 496/979] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:51:23 2022] GET /identity/v3/role_assignments?user.id=1bac9dc42db9451e9e87a88c17f29c80&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=641561cf524144e5a49f595528477389 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:23.273441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfb2f-ca6f-4a5a-b378-ce16afd6c431 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:23.273980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfb2f-ca6f-4a5a-b378-ce16afd6c431 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:23.274224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfb2f-ca6f-4a5a-b378-ce16afd6c431 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:23.275612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 484/980] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:23.281696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d687fb-79cc-4dbf-aa4a-bad9efbe5b3c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:23.282216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d687fb-79cc-4dbf-aa4a-bad9efbe5b3c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:23.282292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d687fb-79cc-4dbf-aa4a-bad9efbe5b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:23.301894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-41d687fb-79cc-4dbf-aa4a-bad9efbe5b3c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:23.303139 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-41d687fb-79cc-4dbf-aa4a-bad9efbe5b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:23.380976 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 497/981] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:23.648991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abfb2f-ca6f-4a5a-b378-ce16afd6c431 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:23.690794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2061a-b677-4a08-a94b-b9e490dc9d4e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:23.691133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2061a-b677-4a08-a94b-b9e490dc9d4e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:23.691346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2061a-b677-4a08-a94b-b9e490dc9d4e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:23.693424 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 498/982] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:23.697882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628b9e-0589-4891-9ba6-a1e2212eb65d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:23.698247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628b9e-0589-4891-9ba6-a1e2212eb65d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:23.698519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628b9e-0589-4891-9ba6-a1e2212eb65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:23.716448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-57628b9e-0589-4891-9ba6-a1e2212eb65d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:23.717618 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-57628b9e-0589-4891-9ba6-a1e2212eb65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:23.749183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:23.762940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abfb2f-ca6f-4a5a-b378-ce16afd6c431 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:23.781312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:23.800096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 499/983] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:23.810489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408642d7-6570-4907-9a25-ec1b6dea55a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:23.811091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408642d7-6570-4907-9a25-ec1b6dea55a7 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:23.811370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408642d7-6570-4907-9a25-ec1b6dea55a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:23.811640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408642d7-6570-4907-9a25-ec1b6dea55a7 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:23.812324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408642d7-6570-4907-9a25-ec1b6dea55a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:23.814322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408642d7-6570-4907-9a25-ec1b6dea55a7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:23.829762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 485/984] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:51:23 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:51:24.070020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628b9e-0589-4891-9ba6-a1e2212eb65d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:24.174402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628b9e-0589-4891-9ba6-a1e2212eb65d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:24.220149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73933041-4178-4c27-b2a1-27198fee3f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:24.220807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73933041-4178-4c27-b2a1-27198fee3f58 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:24.221105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73933041-4178-4c27-b2a1-27198fee3f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:24.221376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73933041-4178-4c27-b2a1-27198fee3f58 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:24.221957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73933041-4178-4c27-b2a1-27198fee3f58 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:24.223322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73933041-4178-4c27-b2a1-27198fee3f58 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-73933041-4178-4c27-b2a1-27198fee3f58 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:24.235848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:24.236939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:24.237784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:24.240128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 500/985] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Aug 24 09:51:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:24.243128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408642d7-6570-4907-9a25-ec1b6dea55a7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:24.353856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408642d7-6570-4907-9a25-ec1b6dea55a7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:24.399152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89a0f6-0326-4828-ab4d-63892896dc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:24.399757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89a0f6-0326-4828-ab4d-63892896dc34 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:24.400037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89a0f6-0326-4828-ab4d-63892896dc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:24.400337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89a0f6-0326-4828-ab4d-63892896dc34 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:24.401030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89a0f6-0326-4828-ab4d-63892896dc34 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:24.401337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89a0f6-0326-4828-ab4d-63892896dc34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:24.402709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89a0f6-0326-4828-ab4d-63892896dc34 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:24.412337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 486/986] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Aug 24 09:51:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:24.413489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73933041-4178-4c27-b2a1-27198fee3f58 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:24.445693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:24.520338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73933041-4178-4c27-b2a1-27198fee3f58 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:24.554087 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26fa7a-32bd-48bf-8150-95dc00563b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:24.554730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26fa7a-32bd-48bf-8150-95dc00563b47 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:24.555162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26fa7a-32bd-48bf-8150-95dc00563b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:24.555447 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26fa7a-32bd-48bf-8150-95dc00563b47 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:24.556097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26fa7a-32bd-48bf-8150-95dc00563b47 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:24.556408 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26fa7a-32bd-48bf-8150-95dc00563b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:24.558119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26fa7a-32bd-48bf-8150-95dc00563b47 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:24.567719 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 501/987] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Aug 24 09:51:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:24.568887 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89a0f6-0326-4828-ab4d-63892896dc34 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:24.612689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:24.677080 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89a0f6-0326-4828-ab4d-63892896dc34 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:24.712354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecdbcb-a9c2-4d13-b5f0-8de037b4be57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:24.712955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecdbcb-a9c2-4d13-b5f0-8de037b4be57 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:24.713237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecdbcb-a9c2-4d13-b5f0-8de037b4be57 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:24.713511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecdbcb-a9c2-4d13-b5f0-8de037b4be57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:24.714286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecdbcb-a9c2-4d13-b5f0-8de037b4be57 None admin] RBAC: Adding query filter params (interface=admin, service_id=88b28e796f7e494db9d691ef3c8182f4, region_id=RegionOne) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:24.714602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecdbcb-a9c2-4d13-b5f0-8de037b4be57 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:24.715958 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecdbcb-a9c2-4d13-b5f0-8de037b4be57 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:24.726084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 487/988] 10.222.0.86 () {56 vars in 1249 bytes} [Wed Aug 24 09:51:24 2022] GET /identity/v3/endpoints?service_id=88b28e796f7e494db9d691ef3c8182f4&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:24.727615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26fa7a-32bd-48bf-8150-95dc00563b47 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:24.831833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:24.841790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26fa7a-32bd-48bf-8150-95dc00563b47 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:24.881990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d41041-16e6-4d30-96cd-b9497c5d0622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:24.882517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d41041-16e6-4d30-96cd-b9497c5d0622 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:24.882799 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d41041-16e6-4d30-96cd-b9497c5d0622 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:24.883074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d41041-16e6-4d30-96cd-b9497c5d0622 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:24.883684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d41041-16e6-4d30-96cd-b9497c5d0622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:24.885092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d41041-16e6-4d30-96cd-b9497c5d0622 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:24.900140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 502/989] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Aug 24 09:51:24 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:51:25.197077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d627f-6dbb-49b5-bed3-334545847f17 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:25.197447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d627f-6dbb-49b5-bed3-334545847f17 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:25.197648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d627f-6dbb-49b5-bed3-334545847f17 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:25.199115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 488/990] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:25.204219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8e27d-dcc3-4ce1-aa59-040c6016ddbc None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:25.204509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8e27d-dcc3-4ce1-aa59-040c6016ddbc None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:25.204780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8e27d-dcc3-4ce1-aa59-040c6016ddbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:25.222603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4cd8e27d-dcc3-4ce1-aa59-040c6016ddbc None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:25.223779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4cd8e27d-dcc3-4ce1-aa59-040c6016ddbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:25.294175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 503/991] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:25.567439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d627f-6dbb-49b5-bed3-334545847f17 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:25.672531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d627f-6dbb-49b5-bed3-334545847f17 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:25.674076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:25.708074 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e0e69-ee2a-41d9-9331-4e0a06e2c5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:25.708699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e0e69-ee2a-41d9-9331-4e0a06e2c5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:25.708982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e0e69-ee2a-41d9-9331-4e0a06e2c5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:25.709283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e0e69-ee2a-41d9-9331-4e0a06e2c5f7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:25.709883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e0e69-ee2a-41d9-9331-4e0a06e2c5f7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:25.711325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e0e69-ee2a-41d9-9331-4e0a06e2c5f7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c20e0e69-ee2a-41d9-9331-4e0a06e2c5f7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:25.720192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:25.721182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:51:25.722070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:25.723460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 489/992] 10.222.0.86 () {56 vars in 1105 bytes} [Wed Aug 24 09:51:25 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:25.727121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8e27d-dcc3-4ce1-aa59-040c6016ddbc None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:25.743632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f256551-a870-4362-a264-2291c5b55dba None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:25.743924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f256551-a870-4362-a264-2291c5b55dba None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:25.744276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f256551-a870-4362-a264-2291c5b55dba None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:25.745648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 490/993] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:25.751366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6501b3d-1088-4323-85c4-c25726e167b4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:25.751653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6501b3d-1088-4323-85c4-c25726e167b4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:25.751927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6501b3d-1088-4323-85c4-c25726e167b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:25.771970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e6501b3d-1088-4323-85c4-c25726e167b4 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:25.773643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e6501b3d-1088-4323-85c4-c25726e167b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:25.820724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:25.836631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8e27d-dcc3-4ce1-aa59-040c6016ddbc None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:25.848085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 491/994] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:25.876408 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b042d1-491c-485e-b5ad-7adb599a3e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:25.877017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b042d1-491c-485e-b5ad-7adb599a3e2f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:25.877294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b042d1-491c-485e-b5ad-7adb599a3e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:25.877571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b042d1-491c-485e-b5ad-7adb599a3e2f None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:25.878200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b042d1-491c-485e-b5ad-7adb599a3e2f None admin] RBAC: Adding query filter params (name=image) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:25.878502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b042d1-491c-485e-b5ad-7adb599a3e2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:25.879875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b042d1-491c-485e-b5ad-7adb599a3e2f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:25.885005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 504/995] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:25.890488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6501b3d-1088-4323-85c4-c25726e167b4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:25.988729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:25.995154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6501b3d-1088-4323-85c4-c25726e167b4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:26.031027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad069c-9de6-40ff-9415-3e3cfa1ea78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:26.031630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad069c-9de6-40ff-9415-3e3cfa1ea78b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:26.031935 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad069c-9de6-40ff-9415-3e3cfa1ea78b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:26.032258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad069c-9de6-40ff-9415-3e3cfa1ea78b None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:26.032889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad069c-9de6-40ff-9415-3e3cfa1ea78b None admin] RBAC: Adding query filter params (type=image) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:26.033195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad069c-9de6-40ff-9415-3e3cfa1ea78b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:26.034592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad069c-9de6-40ff-9415-3e3cfa1ea78b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:26.048340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 492/996] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:26.148531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b042d1-491c-485e-b5ad-7adb599a3e2f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:26.254825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b042d1-491c-485e-b5ad-7adb599a3e2f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:26.288722 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088980b-a2b5-452f-81f4-a4aa8dc57bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:26.289348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088980b-a2b5-452f-81f4-a4aa8dc57bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:26.289633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088980b-a2b5-452f-81f4-a4aa8dc57bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:26.289909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088980b-a2b5-452f-81f4-a4aa8dc57bb3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:26.290492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088980b-a2b5-452f-81f4-a4aa8dc57bb3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:26.291867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088980b-a2b5-452f-81f4-a4aa8dc57bb3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-b088980b-a2b5-452f-81f4-a4aa8dc57bb3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:26.301119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:26.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:51:26.302995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:26.304357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 505/997] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:26.308868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad069c-9de6-40ff-9415-3e3cfa1ea78b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:26.412124 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad069c-9de6-40ff-9415-3e3cfa1ea78b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:26.446690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b262a41-4c4b-4798-a9ea-300cfd494174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:26.447284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b262a41-4c4b-4798-a9ea-300cfd494174 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:26.447562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b262a41-4c4b-4798-a9ea-300cfd494174 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:26.447829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b262a41-4c4b-4798-a9ea-300cfd494174 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:26.448477 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b262a41-4c4b-4798-a9ea-300cfd494174 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:26.448778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b262a41-4c4b-4798-a9ea-300cfd494174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:26.450141 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b262a41-4c4b-4798-a9ea-300cfd494174 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:26.459572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 493/998] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:51:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:26.461304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088980b-a2b5-452f-81f4-a4aa8dc57bb3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:26.479565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:26.568647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088980b-a2b5-452f-81f4-a4aa8dc57bb3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:26.603322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed6289-5f66-46dd-a40d-4be98df31315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:26.603912 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed6289-5f66-46dd-a40d-4be98df31315 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:26.604216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed6289-5f66-46dd-a40d-4be98df31315 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:26.604484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed6289-5f66-46dd-a40d-4be98df31315 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:26.605105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed6289-5f66-46dd-a40d-4be98df31315 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:26.605402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed6289-5f66-46dd-a40d-4be98df31315 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:26.606763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed6289-5f66-46dd-a40d-4be98df31315 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:26.617060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 506/999] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:51:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:26.618618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b262a41-4c4b-4798-a9ea-300cfd494174 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:26.644942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:26.726487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b262a41-4c4b-4798-a9ea-300cfd494174 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:26.779677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a93022-707c-4f47-a8f4-04e422da27be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:26.780320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a93022-707c-4f47-a8f4-04e422da27be None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:26.780598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a93022-707c-4f47-a8f4-04e422da27be None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:26.780869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a93022-707c-4f47-a8f4-04e422da27be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:26.781567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a93022-707c-4f47-a8f4-04e422da27be None admin] RBAC: Adding query filter params (interface=public, service_id=346cc409fb0d491d9663c138735a84bd, region_id=RegionOne) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:26.781872 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a93022-707c-4f47-a8f4-04e422da27be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:26.786292 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a93022-707c-4f47-a8f4-04e422da27be None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:26.794822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 494/1000] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Aug 24 09:51:26 2022] GET /identity/v3/endpoints?service_id=346cc409fb0d491d9663c138735a84bd&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:26.795794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed6289-5f66-46dd-a40d-4be98df31315 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:26.805626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aca7c6-1254-4ecc-86f9-bf678d472c3a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:26.805626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aca7c6-1254-4ecc-86f9-bf678d472c3a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:26.805880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aca7c6-1254-4ecc-86f9-bf678d472c3a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:26.807248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 495/1001] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:26.812167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f345f1d-41df-4fc1-b37c-a2378f2bf075 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:26.812453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f345f1d-41df-4fc1-b37c-a2378f2bf075 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:26.812843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f345f1d-41df-4fc1-b37c-a2378f2bf075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:26.831256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9f345f1d-41df-4fc1-b37c-a2378f2bf075 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:26.832470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9f345f1d-41df-4fc1-b37c-a2378f2bf075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:26.882074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:26.904500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 496/1002] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:26.905981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed6289-5f66-46dd-a40d-4be98df31315 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:26.941086 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd4701-1305-4669-90ee-6828606087f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:26.941691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd4701-1305-4669-90ee-6828606087f7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:26.941970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd4701-1305-4669-90ee-6828606087f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:26.942242 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd4701-1305-4669-90ee-6828606087f7 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:26.942846 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd4701-1305-4669-90ee-6828606087f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:26.944256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd4701-1305-4669-90ee-6828606087f7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:26.958261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 507/1003] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Aug 24 09:51:26 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:51:27.173538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f345f1d-41df-4fc1-b37c-a2378f2bf075 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:27.192999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:27.275028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f345f1d-41df-4fc1-b37c-a2378f2bf075 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:27.310519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3060b5-965f-4eb0-ba6e-64a86d9c29cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:27.311144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3060b5-965f-4eb0-ba6e-64a86d9c29cd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:27.311423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3060b5-965f-4eb0-ba6e-64a86d9c29cd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:27.311690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3060b5-965f-4eb0-ba6e-64a86d9c29cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:27.312290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3060b5-965f-4eb0-ba6e-64a86d9c29cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:27.313687 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3060b5-965f-4eb0-ba6e-64a86d9c29cd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bd3060b5-965f-4eb0-ba6e-64a86d9c29cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:27.322223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:27.323206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:51:27.325123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:27.326166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 497/1004] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Aug 24 09:51:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:27.329539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd4701-1305-4669-90ee-6828606087f7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:27.431508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd4701-1305-4669-90ee-6828606087f7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:27.466116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055667a-2427-485a-b732-95ee74d56f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:27.466721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055667a-2427-485a-b732-95ee74d56f21 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:27.466998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055667a-2427-485a-b732-95ee74d56f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:27.467274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055667a-2427-485a-b732-95ee74d56f21 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:27.467899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055667a-2427-485a-b732-95ee74d56f21 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:27.468245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055667a-2427-485a-b732-95ee74d56f21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:27.469630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055667a-2427-485a-b732-95ee74d56f21 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:27.479820 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 508/1005] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Aug 24 09:51:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:27.481423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3060b5-965f-4eb0-ba6e-64a86d9c29cd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:27.567493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:27.588015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3060b5-965f-4eb0-ba6e-64a86d9c29cd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:27.623023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b6157-842d-4079-87e8-1738a5856fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:27.623670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b6157-842d-4079-87e8-1738a5856fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:27.623931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b6157-842d-4079-87e8-1738a5856fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:27.624197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b6157-842d-4079-87e8-1738a5856fd4 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:27.624825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b6157-842d-4079-87e8-1738a5856fd4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:27.625128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b6157-842d-4079-87e8-1738a5856fd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:27.626521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b6157-842d-4079-87e8-1738a5856fd4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:27.636226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 498/1006] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Aug 24 09:51:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:27.637058 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055667a-2427-485a-b732-95ee74d56f21 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:27.646793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:27.741207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055667a-2427-485a-b732-95ee74d56f21 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:27.780266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01c55a-9005-4cad-80df-f3160fa59600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:27.780937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c55a-9005-4cad-80df-f3160fa59600 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:27.781285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c55a-9005-4cad-80df-f3160fa59600 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:27.781619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c55a-9005-4cad-80df-f3160fa59600 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:27.782944 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01c55a-9005-4cad-80df-f3160fa59600 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:27.786141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01c55a-9005-4cad-80df-f3160fa59600 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:27.808087 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 509/1007] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 09:51:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:51:27.973362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045480e-e08a-480f-b004-166bb8b681ff None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:27.973669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045480e-e08a-480f-b004-166bb8b681ff None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:27.973910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045480e-e08a-480f-b004-166bb8b681ff None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:27.975505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 499/1008] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:27.981488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938e156-f51e-4723-81ad-06971b35f225 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:27.981781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938e156-f51e-4723-81ad-06971b35f225 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:27.982049 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938e156-f51e-4723-81ad-06971b35f225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:28.001034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d938e156-f51e-4723-81ad-06971b35f225 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:28.002259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d938e156-f51e-4723-81ad-06971b35f225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:28.066022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:28.081473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 510/1009] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:27 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:28.347708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045480e-e08a-480f-b004-166bb8b681ff None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:28.450205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045480e-e08a-480f-b004-166bb8b681ff None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:28.485159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8032d88-1ec0-46d8-8192-f1a800bed789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:28.485831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8032d88-1ec0-46d8-8192-f1a800bed789 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:28.486068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8032d88-1ec0-46d8-8192-f1a800bed789 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:28.486341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8032d88-1ec0-46d8-8192-f1a800bed789 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:28.486993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8032d88-1ec0-46d8-8192-f1a800bed789 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:28.488395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8032d88-1ec0-46d8-8192-f1a800bed789 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:28.504439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 500/1010] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:51:28 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:51:28.898002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021220d7-9e39-4c39-a5ba-20ff4ef5675e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:28.898523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021220d7-9e39-4c39-a5ba-20ff4ef5675e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:28.898523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021220d7-9e39-4c39-a5ba-20ff4ef5675e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:28.899869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 511/1011] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:28.905429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6423c61-d90d-4781-8ac3-2981202c9c6c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:28.905913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6423c61-d90d-4781-8ac3-2981202c9c6c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:28.905989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6423c61-d90d-4781-8ac3-2981202c9c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:28.920318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:28.927990 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d6423c61-d90d-4781-8ac3-2981202c9c6c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:28.929541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d6423c61-d90d-4781-8ac3-2981202c9c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:29.003969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 501/1012] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:28 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:29.273903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021220d7-9e39-4c39-a5ba-20ff4ef5675e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:29.376073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021220d7-9e39-4c39-a5ba-20ff4ef5675e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:29.410869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931f523-65a1-4355-9a26-2c13c2d14886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:29.411492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931f523-65a1-4355-9a26-2c13c2d14886 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:29.411767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931f523-65a1-4355-9a26-2c13c2d14886 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:29.412036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931f523-65a1-4355-9a26-2c13c2d14886 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:29.412633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931f523-65a1-4355-9a26-2c13c2d14886 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:29.414002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931f523-65a1-4355-9a26-2c13c2d14886 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-0931f523-65a1-4355-9a26-2c13c2d14886 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:29.422832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:29.423838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:51:29.425078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:29.428377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 512/1013] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:29.429913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6423c61-d90d-4781-8ac3-2981202c9c6c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:29.532829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6423c61-d90d-4781-8ac3-2981202c9c6c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:29.562227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:29.569058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8af41-5c69-44a7-b9ce-fd399c555f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:29.569668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8af41-5c69-44a7-b9ce-fd399c555f9e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:29.569949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8af41-5c69-44a7-b9ce-fd399c555f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:29.570223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8af41-5c69-44a7-b9ce-fd399c555f9e None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:29.570855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8af41-5c69-44a7-b9ce-fd399c555f9e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:29.571158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8af41-5c69-44a7-b9ce-fd399c555f9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:29.572864 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8af41-5c69-44a7-b9ce-fd399c555f9e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:29.578766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 502/1014] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:51:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:29.584391 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931f523-65a1-4355-9a26-2c13c2d14886 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:29.645585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:29.694797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931f523-65a1-4355-9a26-2c13c2d14886 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:29.728816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c40d14-3544-4a92-808e-b6d033b6b6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:29.729421 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c40d14-3544-4a92-808e-b6d033b6b6aa None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:29.729702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c40d14-3544-4a92-808e-b6d033b6b6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:29.729978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c40d14-3544-4a92-808e-b6d033b6b6aa None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:29.730638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c40d14-3544-4a92-808e-b6d033b6b6aa None admin] RBAC: Adding query filter params (type=compute) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:29.730946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c40d14-3544-4a92-808e-b6d033b6b6aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:29.732343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c40d14-3544-4a92-808e-b6d033b6b6aa None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:29.742404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 513/1015] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Aug 24 09:51:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:29.743409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8af41-5c69-44a7-b9ce-fd399c555f9e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:29.751323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9066e-618c-4dd2-92cb-32f33c01eb09 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:29.752151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9066e-618c-4dd2-92cb-32f33c01eb09 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:29.752430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9066e-618c-4dd2-92cb-32f33c01eb09 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:29.753866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 514/1016] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:29.758531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b09b-8a0f-4d45-b0cb-8ccf13051af7 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:29.758808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b09b-8a0f-4d45-b0cb-8ccf13051af7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:29.759238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b09b-8a0f-4d45-b0cb-8ccf13051af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:29.780686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cbf5b09b-8a0f-4d45-b0cb-8ccf13051af7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:29.781878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cbf5b09b-8a0f-4d45-b0cb-8ccf13051af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:29.848906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 515/1017] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:29 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:29.858479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8af41-5c69-44a7-b9ce-fd399c555f9e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:29.893971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f9d0e-1ca5-4943-ab9b-c8c345ebe1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:29.894943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9d0e-1ca5-4943-ab9b-c8c345ebe1df None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:29.895428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9d0e-1ca5-4943-ab9b-c8c345ebe1df None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:29.895946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9d0e-1ca5-4943-ab9b-c8c345ebe1df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:29.896831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f9d0e-1ca5-4943-ab9b-c8c345ebe1df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:29.898423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f9d0e-1ca5-4943-ab9b-c8c345ebe1df None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:29.920441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 503/1018] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 09:51:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:51:30.115626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5b09b-8a0f-4d45-b0cb-8ccf13051af7 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:30.168038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:30.223176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5b09b-8a0f-4d45-b0cb-8ccf13051af7 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:30.266664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e78c7-e4f1-47aa-9931-86f10f5a56f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:30.267321 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e78c7-e4f1-47aa-9931-86f10f5a56f9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:30.267596 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e78c7-e4f1-47aa-9931-86f10f5a56f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:30.267870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e78c7-e4f1-47aa-9931-86f10f5a56f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:30.268979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e78c7-e4f1-47aa-9931-86f10f5a56f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:30.270404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e78c7-e4f1-47aa-9931-86f10f5a56f9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:30.272162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 516/1019] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:30.277790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f9d0e-1ca5-4943-ab9b-c8c345ebe1df None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:30.332152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:30.383619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f9d0e-1ca5-4943-ab9b-c8c345ebe1df None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:30.419105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11914e5d-7045-4b4b-96ae-15bc0207dacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:30.419712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914e5d-7045-4b4b-96ae-15bc0207dacb None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:30.419985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914e5d-7045-4b4b-96ae-15bc0207dacb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:30.420299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914e5d-7045-4b4b-96ae-15bc0207dacb None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:30.420949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11914e5d-7045-4b4b-96ae-15bc0207dacb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:30.422373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11914e5d-7045-4b4b-96ae-15bc0207dacb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:30.424255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4a385-9d08-4fb2-a5a0-cad22b2632be None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:30.424611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4a385-9d08-4fb2-a5a0-cad22b2632be None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:30.424902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4a385-9d08-4fb2-a5a0-cad22b2632be None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:30.426465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 517/1020] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:30.431482 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f07638-fa26-4577-9a75-34f68fcb42b4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:30.431766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f07638-fa26-4577-9a75-34f68fcb42b4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:30.432072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f07638-fa26-4577-9a75-34f68fcb42b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:30.450684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-23f07638-fa26-4577-9a75-34f68fcb42b4 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:30.451998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-23f07638-fa26-4577-9a75-34f68fcb42b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:30.458942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 504/1021] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:51:30 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:51:30.529831 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 518/1022] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:30.794396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11914e5d-7045-4b4b-96ae-15bc0207dacb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:30.894366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11914e5d-7045-4b4b-96ae-15bc0207dacb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:30.896755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:30.933544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7df0d-0a21-4927-b80b-063af8e9cf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:30.934182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7df0d-0a21-4927-b80b-063af8e9cf54 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:30.934433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7df0d-0a21-4927-b80b-063af8e9cf54 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:30.934705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7df0d-0a21-4927-b80b-063af8e9cf54 None admin] PATH_INFO: `/v3/services/image` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:30.935286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7df0d-0a21-4927-b80b-063af8e9cf54 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:30.936686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7df0d-0a21-4927-b80b-063af8e9cf54 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a4a7df0d-0a21-4927-b80b-063af8e9cf54 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:30.946284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:30.947325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:51:30.948599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:30.952136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 505/1023] 10.222.0.86 () {56 vars in 1105 bytes} [Wed Aug 24 09:51:30 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:30.953740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f07638-fa26-4577-9a75-34f68fcb42b4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:31.001912 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:31.057987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f07638-fa26-4577-9a75-34f68fcb42b4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:31.093945 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925491d3-0876-4c63-a24a-630f676c097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:31.094612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925491d3-0876-4c63-a24a-630f676c097e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:31.094896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925491d3-0876-4c63-a24a-630f676c097e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:31.095192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925491d3-0876-4c63-a24a-630f676c097e None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:31.095832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925491d3-0876-4c63-a24a-630f676c097e None admin] RBAC: Adding query filter params (name=image) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:31.096167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925491d3-0876-4c63-a24a-630f676c097e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:31.097537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925491d3-0876-4c63-a24a-630f676c097e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:31.108024 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 519/1024] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:31.109405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7df0d-0a21-4927-b80b-063af8e9cf54 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:31.214689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7df0d-0a21-4927-b80b-063af8e9cf54 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:31.249826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51073df9-1f58-4c5f-8259-39cc0736ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:31.250426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073df9-1f58-4c5f-8259-39cc0736ff12 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:31.250707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073df9-1f58-4c5f-8259-39cc0736ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:31.250982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073df9-1f58-4c5f-8259-39cc0736ff12 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:31.252227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51073df9-1f58-4c5f-8259-39cc0736ff12 None admin] RBAC: Adding query filter params (type=image) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:31.252538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51073df9-1f58-4c5f-8259-39cc0736ff12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:31.253937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51073df9-1f58-4c5f-8259-39cc0736ff12 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:31.263847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 506/1025] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:31.265103 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925491d3-0876-4c63-a24a-630f676c097e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:31.368249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925491d3-0876-4c63-a24a-630f676c097e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:31.377953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:31.405384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2c3f3-f2bf-42c0-b59c-fa1d22bfeb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:31.406001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c3f3-f2bf-42c0-b59c-fa1d22bfeb1d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:31.406281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c3f3-f2bf-42c0-b59c-fa1d22bfeb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:31.406552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c3f3-f2bf-42c0-b59c-fa1d22bfeb1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:31.407253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2c3f3-f2bf-42c0-b59c-fa1d22bfeb1d None admin] RBAC: Adding query filter params (interface=public, service_id=1641eab6ff884787bfc7d1b73df7bff2, region_id=RegionOne) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:31.407563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2c3f3-f2bf-42c0-b59c-fa1d22bfeb1d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:31.409514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2c3f3-f2bf-42c0-b59c-fa1d22bfeb1d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:31.415142 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 520/1026] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Aug 24 09:51:31 2022] GET /identity/v3/endpoints?service_id=1641eab6ff884787bfc7d1b73df7bff2&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:31.421274 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51073df9-1f58-4c5f-8259-39cc0736ff12 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:31.454507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:31.538515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51073df9-1f58-4c5f-8259-39cc0736ff12 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:31.573410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963c8bb-ca4e-4de2-ad31-d6cd08c5d505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:31.574010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963c8bb-ca4e-4de2-ad31-d6cd08c5d505 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:31.574288 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963c8bb-ca4e-4de2-ad31-d6cd08c5d505 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:31.574558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963c8bb-ca4e-4de2-ad31-d6cd08c5d505 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:31.575157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963c8bb-ca4e-4de2-ad31-d6cd08c5d505 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:31.576557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963c8bb-ca4e-4de2-ad31-d6cd08c5d505 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:31.583829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 507/1027] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Aug 24 09:51:31 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:51:31.954566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014e758-bd00-4104-87cf-35490b7e1643 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:31.955434 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014e758-bd00-4104-87cf-35490b7e1643 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:31.955659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014e758-bd00-4104-87cf-35490b7e1643 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:31.957098 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 521/1028] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:31.962164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39397dc8-7305-4974-b4e7-86092132e761 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:31.962598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39397dc8-7305-4974-b4e7-86092132e761 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:31.962761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39397dc8-7305-4974-b4e7-86092132e761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:31.981251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-39397dc8-7305-4974-b4e7-86092132e761 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:31.982430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-39397dc8-7305-4974-b4e7-86092132e761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:32.050953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 508/1029] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:32.320707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014e758-bd00-4104-87cf-35490b7e1643 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:32.425157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014e758-bd00-4104-87cf-35490b7e1643 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:32.446651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f147d-9410-4f09-a236-bc56e4a1caa4 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:32.446945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f147d-9410-4f09-a236-bc56e4a1caa4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:32.447219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f147d-9410-4f09-a236-bc56e4a1caa4 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:32.448612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 509/1030] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:32.453662 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98418d-88e6-40a9-b403-4db852194447 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:32.453947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98418d-88e6-40a9-b403-4db852194447 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:32.454222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98418d-88e6-40a9-b403-4db852194447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:32.459924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a15ca-9df3-449d-8b65-64e9b985d0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:32.460570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a15ca-9df3-449d-8b65-64e9b985d0bd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:32.460843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a15ca-9df3-449d-8b65-64e9b985d0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:32.461108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a15ca-9df3-449d-8b65-64e9b985d0bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:32.472368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a15ca-9df3-449d-8b65-64e9b985d0bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:32.472512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5e98418d-88e6-40a9-b403-4db852194447 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:32.473627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5e98418d-88e6-40a9-b403-4db852194447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-599a15ca-9df3-449d-8b65-64e9b985d0bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:32.477875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:32.478851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:32.479807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:32.483968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 522/1031] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Aug 24 09:51:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:32.484809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a15ca-9df3-449d-8b65-64e9b985d0bd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:32.501507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:32.520503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:32.554954 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 510/1032] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:32.599010 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a15ca-9df3-449d-8b65-64e9b985d0bd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:32.633969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84ee46-038b-4611-af40-4ca7d13d1d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:32.634571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84ee46-038b-4611-af40-4ca7d13d1d52 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:32.634851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84ee46-038b-4611-af40-4ca7d13d1d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:32.635131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84ee46-038b-4611-af40-4ca7d13d1d52 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:32.635792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84ee46-038b-4611-af40-4ca7d13d1d52 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:32.636132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84ee46-038b-4611-af40-4ca7d13d1d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:32.637502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84ee46-038b-4611-af40-4ca7d13d1d52 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:32.650671 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 523/1033] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Aug 24 09:51:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:32.651849 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98418d-88e6-40a9-b403-4db852194447 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:32.755002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98418d-88e6-40a9-b403-4db852194447 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:32.797293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1e491-2f92-4294-a7e8-e023a266dc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:32.798147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1e491-2f92-4294-a7e8-e023a266dc2f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:32.798427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1e491-2f92-4294-a7e8-e023a266dc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:32.798730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1e491-2f92-4294-a7e8-e023a266dc2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:32.809648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1e491-2f92-4294-a7e8-e023a266dc2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:32.811083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1e491-2f92-4294-a7e8-e023a266dc2f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8cf1e491-2f92-4294-a7e8-e023a266dc2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:32.825702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:32.826721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:32.827648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:32.827648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:32.827648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:32.827648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 511/1034] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:32.827890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84ee46-038b-4611-af40-4ca7d13d1d52 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:32.836845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1e491-2f92-4294-a7e8-e023a266dc2f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:32.851549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:32.937995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:32.946571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84ee46-038b-4611-af40-4ca7d13d1d52 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:32.949473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1e491-2f92-4294-a7e8-e023a266dc2f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:32.984599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14c5e-5f7a-462a-ad07-8d34ba55d60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:32.985235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14c5e-5f7a-462a-ad07-8d34ba55d60d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:32.985452 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14c5e-5f7a-462a-ad07-8d34ba55d60d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:32.985723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14c5e-5f7a-462a-ad07-8d34ba55d60d None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:32.986387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14c5e-5f7a-462a-ad07-8d34ba55d60d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:32.986720 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14c5e-5f7a-462a-ad07-8d34ba55d60d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:32.988059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14c5e-5f7a-462a-ad07-8d34ba55d60d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:32.988688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056285c-0479-4899-9ca6-a14ad5b39b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:32.989578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056285c-0479-4899-9ca6-a14ad5b39b1f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:32.990052 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056285c-0479-4899-9ca6-a14ad5b39b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:32.990542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056285c-0479-4899-9ca6-a14ad5b39b1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:32.991669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056285c-0479-4899-9ca6-a14ad5b39b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:32.993123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056285c-0479-4899-9ca6-a14ad5b39b1f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:32.995408 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 524/1035] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:33.000553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 512/1036] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:33.000880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056285c-0479-4899-9ca6-a14ad5b39b1f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:33.005823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14c5e-5f7a-462a-ad07-8d34ba55d60d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:33.078318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:33.110628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056285c-0479-4899-9ca6-a14ad5b39b1f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:33.114857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14c5e-5f7a-462a-ad07-8d34ba55d60d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:33.153607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60166f-4ad1-4d25-ae03-92baece19630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:33.154225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60166f-4ad1-4d25-ae03-92baece19630 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:33.154506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60166f-4ad1-4d25-ae03-92baece19630 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:33.154777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60166f-4ad1-4d25-ae03-92baece19630 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:33.160824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1d812-6138-41d4-bafe-b126380098d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:33.161818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1d812-6138-41d4-bafe-b126380098d0 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:33.162312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1d812-6138-41d4-bafe-b126380098d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:33.162784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1d812-6138-41d4-bafe-b126380098d0 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:33.163278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60166f-4ad1-4d25-ae03-92baece19630 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:33.164612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1d812-6138-41d4-bafe-b126380098d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:33.164846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60166f-4ad1-4d25-ae03-92baece19630 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:33.165535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1d812-6138-41d4-bafe-b126380098d0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0e60166f-4ad1-4d25-ae03-92baece19630 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:33.177730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:33.178690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:33.180059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:33.180059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:33.180059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:33.180059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:33.180059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:33.180059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:33.180059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:33.180059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:33.180059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:51:33.180059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:33.186755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 513/1037] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Aug 24 09:51:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:33.188032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60166f-4ad1-4d25-ae03-92baece19630 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:33.195304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 525/1038] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:51:32 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:51:33.308133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60166f-4ad1-4d25-ae03-92baece19630 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:33.337883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:33.349037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63b569-5627-4bf6-9e13-d7f670c19846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:33.349647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63b569-5627-4bf6-9e13-d7f670c19846 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:33.349929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63b569-5627-4bf6-9e13-d7f670c19846 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:33.350205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63b569-5627-4bf6-9e13-d7f670c19846 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:33.350939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63b569-5627-4bf6-9e13-d7f670c19846 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:33.351244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63b569-5627-4bf6-9e13-d7f670c19846 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:33.352707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63b569-5627-4bf6-9e13-d7f670c19846 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:33.370657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 514/1039] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Aug 24 09:51:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:33.376562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1d812-6138-41d4-bafe-b126380098d0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:33.399904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:33.488032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1d812-6138-41d4-bafe-b126380098d0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:33.523274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad8db4-1e69-42a4-96b0-7559f6b528d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:33.523887 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad8db4-1e69-42a4-96b0-7559f6b528d4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:33.524352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad8db4-1e69-42a4-96b0-7559f6b528d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:33.524554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad8db4-1e69-42a4-96b0-7559f6b528d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:33.526441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1f783-e0b7-4584-add4-c8c3ea30ffeb None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:33.526441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1f783-e0b7-4584-add4-c8c3ea30ffeb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:33.526441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1f783-e0b7-4584-add4-c8c3ea30ffeb None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:33.527944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 515/1040] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:33.533001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bc266-ef2b-46e2-aa89-26be4e7d2dac None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:33.533489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bc266-ef2b-46e2-aa89-26be4e7d2dac None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:33.533761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bc266-ef2b-46e2-aa89-26be4e7d2dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:33.551739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d34bc266-ef2b-46e2-aa89-26be4e7d2dac None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:33.552937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d34bc266-ef2b-46e2-aa89-26be4e7d2dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:33.597600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad8db4-1e69-42a4-96b0-7559f6b528d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:33.599159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad8db4-1e69-42a4-96b0-7559f6b528d4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:33.620484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:33.622947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 516/1041] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:33.707257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 526/1042] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:33 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:51:33.708545 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bc266-ef2b-46e2-aa89-26be4e7d2dac None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:33.819122 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bc266-ef2b-46e2-aa89-26be4e7d2dac None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:33.835882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:33.855099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e3de8-bb8d-4fae-a75e-b4e267461cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:33.855734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e3de8-bb8d-4fae-a75e-b4e267461cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:33.856013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e3de8-bb8d-4fae-a75e-b4e267461cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:33.856309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e3de8-bb8d-4fae-a75e-b4e267461cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:33.867089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e3de8-bb8d-4fae-a75e-b4e267461cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:33.868565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e3de8-bb8d-4fae-a75e-b4e267461cb3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-038e3de8-bb8d-4fae-a75e-b4e267461cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:33.882965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:33.883909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:33.885934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:33.885934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:33.885934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:33.885934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 517/1043] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:33.889714 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad8db4-1e69-42a4-96b0-7559f6b528d4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:33.897343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e3de8-bb8d-4fae-a75e-b4e267461cb3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:33.999195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad8db4-1e69-42a4-96b0-7559f6b528d4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:34.005608 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e3de8-bb8d-4fae-a75e-b4e267461cb3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:34.016235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:34.034845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94142b5e-904b-40e0-9179-715fc7f18a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:34.035573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94142b5e-904b-40e0-9179-715fc7f18a8d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:34.035856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94142b5e-904b-40e0-9179-715fc7f18a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:34.036180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94142b5e-904b-40e0-9179-715fc7f18a8d None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:34.036827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94142b5e-904b-40e0-9179-715fc7f18a8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:34.037135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94142b5e-904b-40e0-9179-715fc7f18a8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:34.038515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94142b5e-904b-40e0-9179-715fc7f18a8d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:34.045862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211ee92-23e0-4b8d-830e-09e65218b828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:34.046811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211ee92-23e0-4b8d-830e-09e65218b828 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:34.047086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211ee92-23e0-4b8d-830e-09e65218b828 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:34.047355 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211ee92-23e0-4b8d-830e-09e65218b828 None admin] PATH_INFO: `/v3/services/image` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:34.047934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211ee92-23e0-4b8d-830e-09e65218b828 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:34.049874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211ee92-23e0-4b8d-830e-09e65218b828 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:34.055179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 527/1044] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:34.056084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94142b5e-904b-40e0-9179-715fc7f18a8d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3211ee92-23e0-4b8d-830e-09e65218b828 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:34.063191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:34.064253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:51:34.065148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:34.068351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 518/1045] 10.222.0.86 () {56 vars in 1105 bytes} [Wed Aug 24 09:51:33 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:34.070128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211ee92-23e0-4b8d-830e-09e65218b828 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:34.108282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:34.164720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94142b5e-904b-40e0-9179-715fc7f18a8d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:34.185924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211ee92-23e0-4b8d-830e-09e65218b828 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:34.200520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6fc43-d9f9-4da0-872c-19f0c493480e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:34.201148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6fc43-d9f9-4da0-872c-19f0c493480e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:34.201425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6fc43-d9f9-4da0-872c-19f0c493480e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:34.201690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6fc43-d9f9-4da0-872c-19f0c493480e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:34.209149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6fc43-d9f9-4da0-872c-19f0c493480e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:34.210587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6fc43-d9f9-4da0-872c-19f0c493480e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-98b6fc43-d9f9-4da0-872c-19f0c493480e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:34.222374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:34.223286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:34.224241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:34.224241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:34.224241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:34.224241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:34.224241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:34.224241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:34.224241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:34.224241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:34.225188 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f345ef-b59c-4041-aef5-6d156b86a87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:34.225188 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f345ef-b59c-4041-aef5-6d156b86a87e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:34.225188 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f345ef-b59c-4041-aef5-6d156b86a87e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:34.225188 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f345ef-b59c-4041-aef5-6d156b86a87e None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:34.225484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f345ef-b59c-4041-aef5-6d156b86a87e None admin] RBAC: Adding query filter params (name=image) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:34.225484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f345ef-b59c-4041-aef5-6d156b86a87e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:34.226141 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f345ef-b59c-4041-aef5-6d156b86a87e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:34.229015 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 528/1046] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:34.230213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6fc43-d9f9-4da0-872c-19f0c493480e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:34.236285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 519/1047] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:34.238145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f345ef-b59c-4041-aef5-6d156b86a87e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:34.276987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:34.343870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:34.344172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6fc43-d9f9-4da0-872c-19f0c493480e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:34.356849 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f345ef-b59c-4041-aef5-6d156b86a87e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:34.382774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73957d49-d860-4ef1-91e8-278bce846642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:34.383384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957d49-d860-4ef1-91e8-278bce846642 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:34.383888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957d49-d860-4ef1-91e8-278bce846642 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:34.383888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957d49-d860-4ef1-91e8-278bce846642 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:34.384627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73957d49-d860-4ef1-91e8-278bce846642 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:34.384905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73957d49-d860-4ef1-91e8-278bce846642 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:34.386420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73957d49-d860-4ef1-91e8-278bce846642 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:34.395548 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a50681-0f44-4d43-83fb-b1e259e1d5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:34.396185 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a50681-0f44-4d43-83fb-b1e259e1d5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:34.396459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a50681-0f44-4d43-83fb-b1e259e1d5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:34.396745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a50681-0f44-4d43-83fb-b1e259e1d5b9 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:34.397487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a50681-0f44-4d43-83fb-b1e259e1d5b9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:34.397798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a50681-0f44-4d43-83fb-b1e259e1d5b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:34.399147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a50681-0f44-4d43-83fb-b1e259e1d5b9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:34.399737 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 529/1048] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:51:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:34.409373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 520/1049] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:34.409546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73957d49-d860-4ef1-91e8-278bce846642 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:34.410287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a50681-0f44-4d43-83fb-b1e259e1d5b9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:34.509186 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:34.514435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73957d49-d860-4ef1-91e8-278bce846642 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:34.523120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a50681-0f44-4d43-83fb-b1e259e1d5b9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:34.550218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964752dc-510d-4943-809f-f09716151911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:34.550809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964752dc-510d-4943-809f-f09716151911 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:34.551085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964752dc-510d-4943-809f-f09716151911 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:34.551605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964752dc-510d-4943-809f-f09716151911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:34.552082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964752dc-510d-4943-809f-f09716151911 None admin] RBAC: Adding query filter params (role.id=641561cf524144e5a49f595528477389, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=98e11312493643b199fca04cb2d77853) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:34.552390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964752dc-510d-4943-809f-f09716151911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:34.553911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964752dc-510d-4943-809f-f09716151911 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:34.568088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 530/1050] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:51:34 2022] GET /identity/v3/role_assignments?user.id=98e11312493643b199fca04cb2d77853&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=641561cf524144e5a49f595528477389 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:34.569283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462f779-1ffc-4e62-8432-48b53b1967c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:34.569928 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462f779-1ffc-4e62-8432-48b53b1967c5 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:34.570209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462f779-1ffc-4e62-8432-48b53b1967c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:34.570486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462f779-1ffc-4e62-8432-48b53b1967c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:34.571177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462f779-1ffc-4e62-8432-48b53b1967c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:34.572667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462f779-1ffc-4e62-8432-48b53b1967c5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:34.604079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 521/1051] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 09:51:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:51:35.202379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381436ab-8f5c-46e4-86f0-34210b9fa2fa None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:35.202744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381436ab-8f5c-46e4-86f0-34210b9fa2fa None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:35.202925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381436ab-8f5c-46e4-86f0-34210b9fa2fa None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:35.204470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 531/1052] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:35.209449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffe758-2f9b-4ec5-915a-14375986a990 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:35.209905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffe758-2f9b-4ec5-915a-14375986a990 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:35.210044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffe758-2f9b-4ec5-915a-14375986a990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:35.228774 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c5ffe758-2f9b-4ec5-915a-14375986a990 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:35.230041 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c5ffe758-2f9b-4ec5-915a-14375986a990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:35.242286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:35.312088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 522/1053] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:35.576627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381436ab-8f5c-46e4-86f0-34210b9fa2fa None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:35.682669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381436ab-8f5c-46e4-86f0-34210b9fa2fa None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:35.718286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d509fde-9679-4d89-b9b1-1c376c793f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:35.718910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d509fde-9679-4d89-b9b1-1c376c793f88 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:35.719184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d509fde-9679-4d89-b9b1-1c376c793f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:35.719452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d509fde-9679-4d89-b9b1-1c376c793f88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:35.730803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d509fde-9679-4d89-b9b1-1c376c793f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-7d509fde-9679-4d89-b9b1-1c376c793f88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:35.735819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:35.738745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:35.739685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:35.742050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 532/1054] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:51:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:35.743309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffe758-2f9b-4ec5-915a-14375986a990 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:35.852513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffe758-2f9b-4ec5-915a-14375986a990 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:35.887028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83e721-1d9b-4c3d-aff2-f00ba9fb92f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:35.887691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e721-1d9b-4c3d-aff2-f00ba9fb92f7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:35.887975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e721-1d9b-4c3d-aff2-f00ba9fb92f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:35.888319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e721-1d9b-4c3d-aff2-f00ba9fb92f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:35.888994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83e721-1d9b-4c3d-aff2-f00ba9fb92f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:35.889298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83e721-1d9b-4c3d-aff2-f00ba9fb92f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:35.890664 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83e721-1d9b-4c3d-aff2-f00ba9fb92f7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:35.899696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 523/1055] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:51:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:35.905253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d509fde-9679-4d89-b9b1-1c376c793f88 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:35.930928 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:36.007965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d509fde-9679-4d89-b9b1-1c376c793f88 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:36.042088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315275b-892e-47d2-b4a9-dc7258ee3af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:36.042570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315275b-892e-47d2-b4a9-dc7258ee3af3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:36.042844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315275b-892e-47d2-b4a9-dc7258ee3af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:36.043113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315275b-892e-47d2-b4a9-dc7258ee3af3 None admin] PATH_INFO: `/v3/users/412b9fd44c064f9b9b77f2dc9655085e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:36.057102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315275b-892e-47d2-b4a9-dc7258ee3af3 None admin] RBAC: Authorizing `identity:get_user(user_id=412b9fd44c064f9b9b77f2dc9655085e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:36.058937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315275b-892e-47d2-b4a9-dc7258ee3af3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:36.060882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 533/1056] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:51:35 2022] GET /identity/v3/users/412b9fd44c064f9b9b77f2dc9655085e => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:36.067238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83e721-1d9b-4c3d-aff2-f00ba9fb92f7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:36.071032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:36.170722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83e721-1d9b-4c3d-aff2-f00ba9fb92f7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:36.204642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31f4aa-066b-45d5-b7a9-23c7ae33f34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:36.205244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31f4aa-066b-45d5-b7a9-23c7ae33f34e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:36.205549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31f4aa-066b-45d5-b7a9-23c7ae33f34e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:36.205818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31f4aa-066b-45d5-b7a9-23c7ae33f34e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:36.275034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31f4aa-066b-45d5-b7a9-23c7ae33f34e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:36.276513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31f4aa-066b-45d5-b7a9-23c7ae33f34e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:36.279915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:36.381144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 524/1057] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:51:36.387403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315275b-892e-47d2-b4a9-dc7258ee3af3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:36.458330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338ba8-547a-4f70-9530-8055f5952929 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:36.458620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338ba8-547a-4f70-9530-8055f5952929 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:36.458885 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338ba8-547a-4f70-9530-8055f5952929 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:36.460406 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 525/1058] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:36.465353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cd6c5-569e-49f0-ba9f-22ab6fb66140 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:36.465639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cd6c5-569e-49f0-ba9f-22ab6fb66140 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:36.465915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cd6c5-569e-49f0-ba9f-22ab6fb66140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:36.473164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:36.484951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-138cd6c5-569e-49f0-ba9f-22ab6fb66140 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:36.486143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-138cd6c5-569e-49f0-ba9f-22ab6fb66140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:36.494645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315275b-892e-47d2-b4a9-dc7258ee3af3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:36.530667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cac1d8-cb3b-40c6-8edc-ff7e93442d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:36.531289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cac1d8-cb3b-40c6-8edc-ff7e93442d11 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:36.531576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cac1d8-cb3b-40c6-8edc-ff7e93442d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:36.531847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cac1d8-cb3b-40c6-8edc-ff7e93442d11 None admin] PATH_INFO: `/v3/projects/8de61c12e65748cca7cd676655bfee3a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:36.533004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cac1d8-cb3b-40c6-8edc-ff7e93442d11 None admin] RBAC: Authorizing `identity:get_project(project_id=8de61c12e65748cca7cd676655bfee3a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:36.534971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cac1d8-cb3b-40c6-8edc-ff7e93442d11 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:36.536815 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 534/1059] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:51:36 2022] GET /identity/v3/projects/8de61c12e65748cca7cd676655bfee3a => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:36.542563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cac1d8-cb3b-40c6-8edc-ff7e93442d11 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:36.559556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 526/1060] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:36.560983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046e2f6-a7c4-4e24-b794-3fccfb99978c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:36.561285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046e2f6-a7c4-4e24-b794-3fccfb99978c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:36.561559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046e2f6-a7c4-4e24-b794-3fccfb99978c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:36.564094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 527/1061] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:36.567825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa8100-ad66-4dc1-8f86-e7ec147b3578 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:36.568182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa8100-ad66-4dc1-8f86-e7ec147b3578 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:36.568418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa8100-ad66-4dc1-8f86-e7ec147b3578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:36.588908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b2fa8100-ad66-4dc1-8f86-e7ec147b3578 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:36.590742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b2fa8100-ad66-4dc1-8f86-e7ec147b3578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:36.623618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:36.667106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cac1d8-cb3b-40c6-8edc-ff7e93442d11 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:36.667779 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 528/1062] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:36.702206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf77f6-e3a8-4bb8-a0c9-631c99f67459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:36.702795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf77f6-e3a8-4bb8-a0c9-631c99f67459 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:36.703073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf77f6-e3a8-4bb8-a0c9-631c99f67459 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:36.703342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf77f6-e3a8-4bb8-a0c9-631c99f67459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:36.704099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf77f6-e3a8-4bb8-a0c9-631c99f67459 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=8de61c12e65748cca7cd676655bfee3a, user.id=412b9fd44c064f9b9b77f2dc9655085e) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:36.704385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf77f6-e3a8-4bb8-a0c9-631c99f67459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:36.706352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf77f6-e3a8-4bb8-a0c9-631c99f67459 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:36.711937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 535/1063] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:51:36 2022] GET /identity/v3/role_assignments?user.id=412b9fd44c064f9b9b77f2dc9655085e&scope.project.id=8de61c12e65748cca7cd676655bfee3a&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:36.830071 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa8100-ad66-4dc1-8f86-e7ec147b3578 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:36.936277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa8100-ad66-4dc1-8f86-e7ec147b3578 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:36.945088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf77f6-e3a8-4bb8-a0c9-631c99f67459 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:36.960917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:36.982646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13acde-2f9f-49dd-aac0-b90e8370bfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:36.983199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13acde-2f9f-49dd-aac0-b90e8370bfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:36.983476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13acde-2f9f-49dd-aac0-b90e8370bfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:36.983753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13acde-2f9f-49dd-aac0-b90e8370bfb0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:36.998076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13acde-2f9f-49dd-aac0-b90e8370bfb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-1d13acde-2f9f-49dd-aac0-b90e8370bfb0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:37.002927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:37.003931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:37.004906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 529/1064] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Aug 24 09:51:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:37.009787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13acde-2f9f-49dd-aac0-b90e8370bfb0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:37.036697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:37.063858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf77f6-e3a8-4bb8-a0c9-631c99f67459 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:37.100865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917ebf6-9f20-4b7d-9473-10258c31687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:37.101466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ebf6-9f20-4b7d-9473-10258c31687f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:37.101749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ebf6-9f20-4b7d-9473-10258c31687f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:37.102020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ebf6-9f20-4b7d-9473-10258c31687f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:37.122652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13acde-2f9f-49dd-aac0-b90e8370bfb0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:37.158117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7062502-b296-44a4-add6-ae9829081450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:37.159254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7062502-b296-44a4-add6-ae9829081450 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:37.159537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7062502-b296-44a4-add6-ae9829081450 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:37.159808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7062502-b296-44a4-add6-ae9829081450 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:37.160502 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7062502-b296-44a4-add6-ae9829081450 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:37.160809 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7062502-b296-44a4-add6-ae9829081450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:37.162178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7062502-b296-44a4-add6-ae9829081450 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:37.170690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 530/1065] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Aug 24 09:51:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:37.176802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7062502-b296-44a4-add6-ae9829081450 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:37.183579 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917ebf6-9f20-4b7d-9473-10258c31687f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:37.185322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917ebf6-9f20-4b7d-9473-10258c31687f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:37.198973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:37.272766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:37.292329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7062502-b296-44a4-add6-ae9829081450 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:37.299356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 536/1066] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:51:37.299356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917ebf6-9f20-4b7d-9473-10258c31687f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:37.330429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303c115-714c-450c-9be4-ebe6bac5c525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:37.331549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c115-714c-450c-9be4-ebe6bac5c525 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:37.331836 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c115-714c-450c-9be4-ebe6bac5c525 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:37.332138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c115-714c-450c-9be4-ebe6bac5c525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:37.343128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303c115-714c-450c-9be4-ebe6bac5c525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:37.344624 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303c115-714c-450c-9be4-ebe6bac5c525 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a303c115-714c-450c-9be4-ebe6bac5c525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:37.360078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:37.361042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:37.362681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:37.362681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:37.362681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:37.362681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 531/1067] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:37.366882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303c115-714c-450c-9be4-ebe6bac5c525 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:37.405037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917ebf6-9f20-4b7d-9473-10258c31687f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:37.418997 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:37.442420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c6667-aa56-4f0d-a728-32029f696a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:37.443044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c6667-aa56-4f0d-a728-32029f696a58 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:37.443327 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c6667-aa56-4f0d-a728-32029f696a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:37.443601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c6667-aa56-4f0d-a728-32029f696a58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:37.458396 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c6667-aa56-4f0d-a728-32029f696a58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:37.460194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c6667-aa56-4f0d-a728-32029f696a58 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:37.475746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303c115-714c-450c-9be4-ebe6bac5c525 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-4d5c6667-aa56-4f0d-a728-32029f696a58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:37.476130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:37.477055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:37.482889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:37.482889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:37.482889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:37.482889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 537/1068] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:37.483141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c6667-aa56-4f0d-a728-32029f696a58 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:37.516857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e98f6-a850-4bf0-b962-d4a08ca1d8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:37.518735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e98f6-a850-4bf0-b962-d4a08ca1d8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:37.519017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e98f6-a850-4bf0-b962-d4a08ca1d8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:37.519290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e98f6-a850-4bf0-b962-d4a08ca1d8d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:37.520116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e98f6-a850-4bf0-b962-d4a08ca1d8d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:37.520423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e98f6-a850-4bf0-b962-d4a08ca1d8d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:37.521795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e98f6-a850-4bf0-b962-d4a08ca1d8d5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:37.531281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:37.538693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 532/1069] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:37.543945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e98f6-a850-4bf0-b962-d4a08ca1d8d5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:37.589879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c6667-aa56-4f0d-a728-32029f696a58 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:37.623793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003b24-1f6b-4e35-ba28-d9fba2feefef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:37.624437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003b24-1f6b-4e35-ba28-d9fba2feefef None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:37.624713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003b24-1f6b-4e35-ba28-d9fba2feefef None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:37.624979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003b24-1f6b-4e35-ba28-d9fba2feefef None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:37.625616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9003b24-1f6b-4e35-ba28-d9fba2feefef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:37.625915 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9003b24-1f6b-4e35-ba28-d9fba2feefef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:37.627285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9003b24-1f6b-4e35-ba28-d9fba2feefef None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:37.639897 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 538/1070] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:37.658327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e98f6-a850-4bf0-b962-d4a08ca1d8d5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:37.698898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf7cb2-ebd2-42b6-92cb-fdfd217d8ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:37.699307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf7cb2-ebd2-42b6-92cb-fdfd217d8ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:37.699583 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf7cb2-ebd2-42b6-92cb-fdfd217d8ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:37.699849 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf7cb2-ebd2-42b6-92cb-fdfd217d8ac1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:37.709072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf7cb2-ebd2-42b6-92cb-fdfd217d8ac1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:37.710559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf7cb2-ebd2-42b6-92cb-fdfd217d8ac1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-87bf7cb2-ebd2-42b6-92cb-fdfd217d8ac1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:37.723581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:37.724941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:37.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:37.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:37.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:37.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:37.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:37.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:37.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:37.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:37.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:51:37.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:37.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 533/1071] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Aug 24 09:51:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:37.732390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003b24-1f6b-4e35-ba28-d9fba2feefef None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:37.763713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:37.851828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003b24-1f6b-4e35-ba28-d9fba2feefef None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:37.889690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915b508-310c-4e64-a335-edad87f26810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:37.890290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915b508-310c-4e64-a335-edad87f26810 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:37.890566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915b508-310c-4e64-a335-edad87f26810 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:37.890832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915b508-310c-4e64-a335-edad87f26810 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:37.891549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915b508-310c-4e64-a335-edad87f26810 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:37.891849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915b508-310c-4e64-a335-edad87f26810 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:37.893938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915b508-310c-4e64-a335-edad87f26810 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:37.911346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 539/1072] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Aug 24 09:51:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:37.917333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf7cb2-ebd2-42b6-92cb-fdfd217d8ac1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:37.977343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:38.019238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf7cb2-ebd2-42b6-92cb-fdfd217d8ac1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:38.052191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36f6e6-e016-49bd-88ac-37a65a6b2038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:38.052816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36f6e6-e016-49bd-88ac-37a65a6b2038 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:38.053093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36f6e6-e016-49bd-88ac-37a65a6b2038 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:38.053363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36f6e6-e016-49bd-88ac-37a65a6b2038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:38.063827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36f6e6-e016-49bd-88ac-37a65a6b2038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:38.065436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36f6e6-e016-49bd-88ac-37a65a6b2038 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7f36f6e6-e016-49bd-88ac-37a65a6b2038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:38.078911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:38.079857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:38.081315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:38.081315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:38.081315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:38.084196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 534/1073] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:38.086352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915b508-310c-4e64-a335-edad87f26810 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:38.167848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:38.186088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915b508-310c-4e64-a335-edad87f26810 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:38.219416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaff338-059a-4995-a5b5-30f8e1d42411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:38.220016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaff338-059a-4995-a5b5-30f8e1d42411 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:38.220323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaff338-059a-4995-a5b5-30f8e1d42411 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:38.220592 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaff338-059a-4995-a5b5-30f8e1d42411 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:38.221232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaff338-059a-4995-a5b5-30f8e1d42411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:38.222075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaff338-059a-4995-a5b5-30f8e1d42411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:38.223439 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaff338-059a-4995-a5b5-30f8e1d42411 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:38.235015 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 540/1074] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:38.241196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36f6e6-e016-49bd-88ac-37a65a6b2038 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:38.343569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36f6e6-e016-49bd-88ac-37a65a6b2038 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:38.354107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:38.378072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9fe04-5be1-4c3c-a8b8-3e67cd585cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:38.378735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9fe04-5be1-4c3c-a8b8-3e67cd585cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:38.379017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9fe04-5be1-4c3c-a8b8-3e67cd585cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:38.379292 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9fe04-5be1-4c3c-a8b8-3e67cd585cb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:38.385835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9fe04-5be1-4c3c-a8b8-3e67cd585cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:38.387270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9fe04-5be1-4c3c-a8b8-3e67cd585cb7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5ac9fe04-5be1-4c3c-a8b8-3e67cd585cb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:38.398024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:38.398894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:38.399956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:38.399956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:38.399956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:38.399956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:38.399956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:38.399956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:38.399956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:38.399956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:38.399956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 535/1075] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:38.404936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaff338-059a-4995-a5b5-30f8e1d42411 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:38.502918 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaff338-059a-4995-a5b5-30f8e1d42411 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:38.535965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25649b-f6c5-4561-b192-1dbd75349523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:38.536602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25649b-f6c5-4561-b192-1dbd75349523 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:38.536883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25649b-f6c5-4561-b192-1dbd75349523 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:38.537160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25649b-f6c5-4561-b192-1dbd75349523 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:38.537852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25649b-f6c5-4561-b192-1dbd75349523 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:38.538162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25649b-f6c5-4561-b192-1dbd75349523 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:38.539596 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25649b-f6c5-4561-b192-1dbd75349523 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:38.552030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 541/1076] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:51:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:38.558148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9fe04-5be1-4c3c-a8b8-3e67cd585cb7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:38.600678 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f54f12-b264-464f-a5dd-7f6ae3775d1b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:38.600958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f54f12-b264-464f-a5dd-7f6ae3775d1b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:38.601233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f54f12-b264-464f-a5dd-7f6ae3775d1b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:38.602630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 542/1077] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:38.607489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ed683-f67d-4274-b173-24953170bad5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:38.607768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ed683-f67d-4274-b173-24953170bad5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:38.608038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ed683-f67d-4274-b173-24953170bad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:38.614891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:38.627490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-822ed683-f67d-4274-b173-24953170bad5 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:38.628757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-822ed683-f67d-4274-b173-24953170bad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:38.662889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9fe04-5be1-4c3c-a8b8-3e67cd585cb7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:38.696460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 543/1078] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:38.696730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ccb98-26e1-44c4-b76e-f1c42f0e4583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:38.697440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ccb98-26e1-44c4-b76e-f1c42f0e4583 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:38.697722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ccb98-26e1-44c4-b76e-f1c42f0e4583 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:38.697992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ccb98-26e1-44c4-b76e-f1c42f0e4583 None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/98e11312493643b199fca04cb2d77853/roles/641561cf524144e5a49f595528477389` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:38.700173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ccb98-26e1-44c4-b76e-f1c42f0e4583 None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=98e11312493643b199fca04cb2d77853, role_id=641561cf524144e5a49f595528477389)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:38.701600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ccb98-26e1-44c4-b76e-f1c42f0e4583 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:38.714802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 536/1079] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:51:38 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/98e11312493643b199fca04cb2d77853/roles/641561cf524144e5a49f595528477389 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:51:38.964538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ed683-f67d-4274-b173-24953170bad5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:39.061694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ed683-f67d-4274-b173-24953170bad5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:39.090292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:39.095856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627d72-fdc3-4f4b-9c5b-598d5eab2a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:39.097059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627d72-fdc3-4f4b-9c5b-598d5eab2a12 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:39.097337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627d72-fdc3-4f4b-9c5b-598d5eab2a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:39.097611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627d72-fdc3-4f4b-9c5b-598d5eab2a12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:39.107763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db627d72-fdc3-4f4b-9c5b-598d5eab2a12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-db627d72-fdc3-4f4b-9c5b-598d5eab2a12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:39.113054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:39.113987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:39.115004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 544/1080] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:51:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:39.120174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ccb98-26e1-44c4-b76e-f1c42f0e4583 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:39.150450 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:39.224728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ccb98-26e1-44c4-b76e-f1c42f0e4583 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:39.257933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22b675-d840-4be0-8a30-553d941711c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:39.258663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22b675-d840-4be0-8a30-553d941711c9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:39.258941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22b675-d840-4be0-8a30-553d941711c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:39.259208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22b675-d840-4be0-8a30-553d941711c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:39.259859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22b675-d840-4be0-8a30-553d941711c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:39.260188 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22b675-d840-4be0-8a30-553d941711c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:39.261542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22b675-d840-4be0-8a30-553d941711c9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:39.269757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 537/1081] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:51:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:39.275516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627d72-fdc3-4f4b-9c5b-598d5eab2a12 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:39.373648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627d72-fdc3-4f4b-9c5b-598d5eab2a12 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:39.406021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc1cab-de56-4cc3-9a96-fd04235b12f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:39.406640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc1cab-de56-4cc3-9a96-fd04235b12f3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:39.406922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc1cab-de56-4cc3-9a96-fd04235b12f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:39.407198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc1cab-de56-4cc3-9a96-fd04235b12f3 None admin] PATH_INFO: `/v3/users/412b9fd44c064f9b9b77f2dc9655085e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:39.408347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc1cab-de56-4cc3-9a96-fd04235b12f3 None admin] RBAC: Authorizing `identity:get_user(user_id=412b9fd44c064f9b9b77f2dc9655085e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:39.410339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc1cab-de56-4cc3-9a96-fd04235b12f3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:39.412107 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 545/1082] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:51:39 2022] GET /identity/v3/users/412b9fd44c064f9b9b77f2dc9655085e => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:39.417838 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22b675-d840-4be0-8a30-553d941711c9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:39.506723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:39.515689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22b675-d840-4be0-8a30-553d941711c9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:39.550191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49680f5c-3532-498c-8591-1f8243eb9c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:39.550805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49680f5c-3532-498c-8591-1f8243eb9c5f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:39.551085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49680f5c-3532-498c-8591-1f8243eb9c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:39.551353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49680f5c-3532-498c-8591-1f8243eb9c5f None admin] PATH_INFO: `/v3/projects/8de61c12e65748cca7cd676655bfee3a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:39.552837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49680f5c-3532-498c-8591-1f8243eb9c5f None admin] RBAC: Authorizing `identity:get_project(project_id=8de61c12e65748cca7cd676655bfee3a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:39.553897 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49680f5c-3532-498c-8591-1f8243eb9c5f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:39.555523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 538/1083] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:51:39 2022] GET /identity/v3/projects/8de61c12e65748cca7cd676655bfee3a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:39.561845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc1cab-de56-4cc3-9a96-fd04235b12f3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:39.620628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:39.661468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc1cab-de56-4cc3-9a96-fd04235b12f3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:39.694921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206a61c-4e78-4724-af35-5d3eb8bf08f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:39.695573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206a61c-4e78-4724-af35-5d3eb8bf08f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:39.695852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206a61c-4e78-4724-af35-5d3eb8bf08f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:39.696158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206a61c-4e78-4724-af35-5d3eb8bf08f5 None admin] PATH_INFO: `/v3/projects/8de61c12e65748cca7cd676655bfee3a/users/412b9fd44c064f9b9b77f2dc9655085e/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:39.698037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206a61c-4e78-4724-af35-5d3eb8bf08f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8de61c12e65748cca7cd676655bfee3a, user_id=412b9fd44c064f9b9b77f2dc9655085e, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:39.699471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206a61c-4e78-4724-af35-5d3eb8bf08f5 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:39.713108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 546/1084] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:51:39 2022] PUT /identity/v3/projects/8de61c12e65748cca7cd676655bfee3a/users/412b9fd44c064f9b9b77f2dc9655085e/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:51:40.600023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dd3db-a4c0-44e7-8954-2e72592b503b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:40.600588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dd3db-a4c0-44e7-8954-2e72592b503b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:40.600588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dd3db-a4c0-44e7-8954-2e72592b503b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:40.601949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 539/1085] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:40.607373 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efa81-f8c7-4b48-b0f0-e0619b3273a7 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:40.607795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efa81-f8c7-4b48-b0f0-e0619b3273a7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:40.607987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efa81-f8c7-4b48-b0f0-e0619b3273a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:40.625793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-450efa81-f8c7-4b48-b0f0-e0619b3273a7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:40.626965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-450efa81-f8c7-4b48-b0f0-e0619b3273a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:40.695772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 547/1086] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:40.962883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19dd3db-a4c0-44e7-8954-2e72592b503b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:41.062262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19dd3db-a4c0-44e7-8954-2e72592b503b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:41.095533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67983aab-8489-465b-91f5-ec0ce584f584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:41.096179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67983aab-8489-465b-91f5-ec0ce584f584 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:41.096457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67983aab-8489-465b-91f5-ec0ce584f584 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:41.096727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67983aab-8489-465b-91f5-ec0ce584f584 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:41.106459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67983aab-8489-465b-91f5-ec0ce584f584 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-67983aab-8489-465b-91f5-ec0ce584f584 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:41.111009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:41.111989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:41.113377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 540/1087] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Aug 24 09:51:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:41.118618 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450efa81-f8c7-4b48-b0f0-e0619b3273a7 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:41.167071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:41.218811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450efa81-f8c7-4b48-b0f0-e0619b3273a7 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:41.252675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0507ae-cc60-4b99-9de5-2d91fdd2b73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:41.253916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0507ae-cc60-4b99-9de5-2d91fdd2b73a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:41.254191 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0507ae-cc60-4b99-9de5-2d91fdd2b73a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:41.254467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0507ae-cc60-4b99-9de5-2d91fdd2b73a None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:41.255131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0507ae-cc60-4b99-9de5-2d91fdd2b73a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:41.255437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0507ae-cc60-4b99-9de5-2d91fdd2b73a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:41.256887 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0507ae-cc60-4b99-9de5-2d91fdd2b73a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:41.264970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 548/1088] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Aug 24 09:51:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:41.270599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67983aab-8489-465b-91f5-ec0ce584f584 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:41.293543 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:41.374284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67983aab-8489-465b-91f5-ec0ce584f584 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:41.406688 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3e9da-0101-4a3b-8792-62a8ccf57d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:41.407304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3e9da-0101-4a3b-8792-62a8ccf57d66 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:41.407581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3e9da-0101-4a3b-8792-62a8ccf57d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:41.407854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3e9da-0101-4a3b-8792-62a8ccf57d66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:41.418745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3e9da-0101-4a3b-8792-62a8ccf57d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:41.420217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3e9da-0101-4a3b-8792-62a8ccf57d66 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-57c3e9da-0101-4a3b-8792-62a8ccf57d66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:41.434510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:41.435731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:41.436692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:41.436692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:41.436692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:41.436692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 541/1089] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:41.441769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0507ae-cc60-4b99-9de5-2d91fdd2b73a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:41.539498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0507ae-cc60-4b99-9de5-2d91fdd2b73a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:41.552359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:41.573160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd5b39-2742-49a7-aa23-1a30af9fc457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:41.573906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd5b39-2742-49a7-aa23-1a30af9fc457 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:41.574036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd5b39-2742-49a7-aa23-1a30af9fc457 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:41.574291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd5b39-2742-49a7-aa23-1a30af9fc457 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:41.574927 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd5b39-2742-49a7-aa23-1a30af9fc457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:41.575228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd5b39-2742-49a7-aa23-1a30af9fc457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:41.576640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd5b39-2742-49a7-aa23-1a30af9fc457 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:41.588279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 549/1090] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:41.594080 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3e9da-0101-4a3b-8792-62a8ccf57d66 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:41.614982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247972-0823-4fe0-8721-cbcdc1894b1a None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:41.615478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247972-0823-4fe0-8721-cbcdc1894b1a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:41.615567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247972-0823-4fe0-8721-cbcdc1894b1a None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:41.616870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 550/1091] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:51:41.621781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912744b3-bbf7-4e1a-9343-162d6d5ce563 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:41.622056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912744b3-bbf7-4e1a-9343-162d6d5ce563 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:41.622328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912744b3-bbf7-4e1a-9343-162d6d5ce563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:41.639789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-912744b3-bbf7-4e1a-9343-162d6d5ce563 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:41.641028 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-912744b3-bbf7-4e1a-9343-162d6d5ce563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:41.666603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:41.700706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3e9da-0101-4a3b-8792-62a8ccf57d66 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:41.709201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 551/1092] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:51:41.734907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ac636-c251-4fb6-ae24-21a938f41b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:41.735529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ac636-c251-4fb6-ae24-21a938f41b5d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:41.735811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ac636-c251-4fb6-ae24-21a938f41b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:41.736240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ac636-c251-4fb6-ae24-21a938f41b5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:41.743558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ac636-c251-4fb6-ae24-21a938f41b5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:41.745023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ac636-c251-4fb6-ae24-21a938f41b5d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ed6ac636-c251-4fb6-ae24-21a938f41b5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:51:41.755982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:41.756934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:41.757893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:41.757893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:41.757893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:51:41.757893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:51:41.757893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:51:41.757893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:51:41.757893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:51:41.757893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:51:41.757893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:51:41.757893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:41.757893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 542/1093] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Aug 24 09:51:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:41.762928 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912744b3-bbf7-4e1a-9343-162d6d5ce563 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:41.863095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912744b3-bbf7-4e1a-9343-162d6d5ce563 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:41.864570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:41.899144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a5913-24f7-456e-9ab3-181f41ca61d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:41.899741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a5913-24f7-456e-9ab3-181f41ca61d7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:41.900024 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a5913-24f7-456e-9ab3-181f41ca61d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:41.900325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a5913-24f7-456e-9ab3-181f41ca61d7 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:41.901066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a5913-24f7-456e-9ab3-181f41ca61d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:41.901375 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a5913-24f7-456e-9ab3-181f41ca61d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:41.902819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a5913-24f7-456e-9ab3-181f41ca61d7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:41.919959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 552/1094] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Aug 24 09:51:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:41.925756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ac636-c251-4fb6-ae24-21a938f41b5d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:41.973363 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a5913-24f7-456e-9ab3-181f41ca61d7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:42.031102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ac636-c251-4fb6-ae24-21a938f41b5d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:42.061574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:42.068238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019b954-6933-4244-9dee-8fb9cbeb5e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:42.068844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b954-6933-4244-9dee-8fb9cbeb5e09 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:42.069128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b954-6933-4244-9dee-8fb9cbeb5e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:42.069403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b954-6933-4244-9dee-8fb9cbeb5e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:42.071539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a5913-24f7-456e-9ab3-181f41ca61d7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:42.104569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91175e86-1aee-431a-b413-014f2bed985c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:42.105181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91175e86-1aee-431a-b413-014f2bed985c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:42.105451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91175e86-1aee-431a-b413-014f2bed985c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:42.105720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91175e86-1aee-431a-b413-014f2bed985c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:42.115668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91175e86-1aee-431a-b413-014f2bed985c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-91175e86-1aee-431a-b413-014f2bed985c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:51:42.120205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:42.121181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:51:42.122060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 553/1095] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:51:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:42.127229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91175e86-1aee-431a-b413-014f2bed985c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:42.136285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019b954-6933-4244-9dee-8fb9cbeb5e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:42.137733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019b954-6933-4244-9dee-8fb9cbeb5e09 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:42.150486 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:42.229468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91175e86-1aee-431a-b413-014f2bed985c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:42.242292 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 543/1096] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:51:42.248360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019b954-6933-4244-9dee-8fb9cbeb5e09 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:42.267593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f69d2-f9c4-4e04-8d8b-44414505ad87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:42.268216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f69d2-f9c4-4e04-8d8b-44414505ad87 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:42.268492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f69d2-f9c4-4e04-8d8b-44414505ad87 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:42.268759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f69d2-f9c4-4e04-8d8b-44414505ad87 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:42.269974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f69d2-f9c4-4e04-8d8b-44414505ad87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:42.270276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f69d2-f9c4-4e04-8d8b-44414505ad87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:42.271638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f69d2-f9c4-4e04-8d8b-44414505ad87 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:42.279949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 554/1097] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:51:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:42.285680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f69d2-f9c4-4e04-8d8b-44414505ad87 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:42.290613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:42.361058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019b954-6933-4244-9dee-8fb9cbeb5e09 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:42.374342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:42.393498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f69d2-f9c4-4e04-8d8b-44414505ad87 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:42.397482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239747d-9fb3-4609-89c6-a0f381df9960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:42.398100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239747d-9fb3-4609-89c6-a0f381df9960 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:42.398376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239747d-9fb3-4609-89c6-a0f381df9960 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:42.398646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239747d-9fb3-4609-89c6-a0f381df9960 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:42.408459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b239747d-9fb3-4609-89c6-a0f381df9960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:42.409887 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b239747d-9fb3-4609-89c6-a0f381df9960 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b239747d-9fb3-4609-89c6-a0f381df9960 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:51:42.423957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:42.425686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:51:42.426659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:51:42.426659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:51:42.426659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:42.426659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 544/1098] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:51:42.428358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d73ab0-3170-434e-b719-50b1a74ed854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:42.428973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73ab0-3170-434e-b719-50b1a74ed854 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:42.429249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73ab0-3170-434e-b719-50b1a74ed854 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:42.429518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73ab0-3170-434e-b719-50b1a74ed854 None admin] PATH_INFO: `/v3/users/412b9fd44c064f9b9b77f2dc9655085e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:42.430648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d73ab0-3170-434e-b719-50b1a74ed854 None admin] RBAC: Authorizing `identity:get_user(user_id=412b9fd44c064f9b9b77f2dc9655085e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:42.431206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239747d-9fb3-4609-89c6-a0f381df9960 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:42.432096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d73ab0-3170-434e-b719-50b1a74ed854 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:42.433977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 555/1099] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:51:42 2022] GET /identity/v3/users/412b9fd44c064f9b9b77f2dc9655085e => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:42.439605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d73ab0-3170-434e-b719-50b1a74ed854 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:42.515242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:42.530712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239747d-9fb3-4609-89c6-a0f381df9960 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:42.540362 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d73ab0-3170-434e-b719-50b1a74ed854 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:42.564349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adef94-9bb7-4714-830b-d9ed58996364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:42.564942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adef94-9bb7-4714-830b-d9ed58996364 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:42.565218 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adef94-9bb7-4714-830b-d9ed58996364 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:42.565487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adef94-9bb7-4714-830b-d9ed58996364 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:42.566714 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adef94-9bb7-4714-830b-d9ed58996364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:42.567015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adef94-9bb7-4714-830b-d9ed58996364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:42.568400 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adef94-9bb7-4714-830b-d9ed58996364 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:42.574556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6b098-a8d5-4c17-bf03-2e1c6aad7164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:42.575153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6b098-a8d5-4c17-bf03-2e1c6aad7164 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:42.575434 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6b098-a8d5-4c17-bf03-2e1c6aad7164 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:42.575700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6b098-a8d5-4c17-bf03-2e1c6aad7164 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:42.581460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 545/1100] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:42.587033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adef94-9bb7-4714-830b-d9ed58996364 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:42.596622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:42.647781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6b098-a8d5-4c17-bf03-2e1c6aad7164 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:42.649192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6b098-a8d5-4c17-bf03-2e1c6aad7164 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:42.688582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adef94-9bb7-4714-830b-d9ed58996364 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:42.722924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cbfc3-4e83-4e0c-bb7c-91a2b42fe8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:42.723551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cbfc3-4e83-4e0c-bb7c-91a2b42fe8ce None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:42.723828 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cbfc3-4e83-4e0c-bb7c-91a2b42fe8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:42.724133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cbfc3-4e83-4e0c-bb7c-91a2b42fe8ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:42.731531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cbfc3-4e83-4e0c-bb7c-91a2b42fe8ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:42.733065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cbfc3-4e83-4e0c-bb7c-91a2b42fe8ce None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bd2cbfc3-4e83-4e0c-bb7c-91a2b42fe8ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:51:42.744410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:51:42.745316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:51:42.746214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:51:42.746214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:51:42.746214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:51:42.746214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:51:42.746214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:51:42.746214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:51:42.746214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:51:42.746214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:51:42.748204 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 546/1101] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:51:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:51:42.749858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 556/1102] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:51:42.751235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cbfc3-4e83-4e0c-bb7c-91a2b42fe8ce None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:42.756129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:42.756252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6b098-a8d5-4c17-bf03-2e1c6aad7164 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:42.804473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:42.855360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6b098-a8d5-4c17-bf03-2e1c6aad7164 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:42.855697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cbfc3-4e83-4e0c-bb7c-91a2b42fe8ce None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:42.889051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef64bd-8c38-4f99-9f7f-d419544420d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:42.889668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef64bd-8c38-4f99-9f7f-d419544420d7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:42.889947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef64bd-8c38-4f99-9f7f-d419544420d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:42.890218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef64bd-8c38-4f99-9f7f-d419544420d7 None admin] PATH_INFO: `/v3/projects/8de61c12e65748cca7cd676655bfee3a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:42.890312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8274b-8e3f-491e-8a3e-a037779478d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:42.890909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8274b-8e3f-491e-8a3e-a037779478d9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:42.891177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8274b-8e3f-491e-8a3e-a037779478d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:42.891384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef64bd-8c38-4f99-9f7f-d419544420d7 None admin] RBAC: Authorizing `identity:get_project(project_id=8de61c12e65748cca7cd676655bfee3a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:42.891477 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8274b-8e3f-491e-8a3e-a037779478d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:42.892159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8274b-8e3f-491e-8a3e-a037779478d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:42.892463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8274b-8e3f-491e-8a3e-a037779478d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:42.893028 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef64bd-8c38-4f99-9f7f-d419544420d7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:42.893928 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8274b-8e3f-491e-8a3e-a037779478d9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:42.894835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 557/1103] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:51:42 2022] GET /identity/v3/projects/8de61c12e65748cca7cd676655bfee3a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:42.901509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef64bd-8c38-4f99-9f7f-d419544420d7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:42.906158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 547/1104] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:51:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:42.912594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8274b-8e3f-491e-8a3e-a037779478d9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:42.974876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:43.003853 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef64bd-8c38-4f99-9f7f-d419544420d7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:43.013656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8274b-8e3f-491e-8a3e-a037779478d9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:43.020540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:43.038220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57119fc-493e-470f-ac86-3839acd5e697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:43.038808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57119fc-493e-470f-ac86-3839acd5e697 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:43.039082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57119fc-493e-470f-ac86-3839acd5e697 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:43.039349 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57119fc-493e-470f-ac86-3839acd5e697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:43.040083 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57119fc-493e-470f-ac86-3839acd5e697 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=8de61c12e65748cca7cd676655bfee3a, user.id=412b9fd44c064f9b9b77f2dc9655085e) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:43.040391 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57119fc-493e-470f-ac86-3839acd5e697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:43.041817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57119fc-493e-470f-ac86-3839acd5e697 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:43.047676 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 558/1105] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:51:42 2022] GET /identity/v3/role_assignments?user.id=412b9fd44c064f9b9b77f2dc9655085e&scope.project.id=8de61c12e65748cca7cd676655bfee3a&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:43.048671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caf1d7-1b45-45fd-985f-7e555ab045ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:43.049275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caf1d7-1b45-45fd-985f-7e555ab045ea None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:43.049553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caf1d7-1b45-45fd-985f-7e555ab045ea None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:43.049824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caf1d7-1b45-45fd-985f-7e555ab045ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:43.051102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86caf1d7-1b45-45fd-985f-7e555ab045ea None admin] RBAC: Adding query filter params (role.id=641561cf524144e5a49f595528477389, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=98e11312493643b199fca04cb2d77853) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:43.051414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86caf1d7-1b45-45fd-985f-7e555ab045ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:43.052871 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86caf1d7-1b45-45fd-985f-7e555ab045ea None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:43.063394 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 548/1106] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:51:42 2022] GET /identity/v3/role_assignments?user.id=98e11312493643b199fca04cb2d77853&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=641561cf524144e5a49f595528477389 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:51:44.969040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18562125-3221-4d88-8f8c-d2e5680aebc4 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:44.969565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18562125-3221-4d88-8f8c-d2e5680aebc4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:44.969565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18562125-3221-4d88-8f8c-d2e5680aebc4 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:44.970925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 559/1107] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:44.976984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf649a5e-25cb-4d60-a26b-2a8cadfd5562 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:44.977398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf649a5e-25cb-4d60-a26b-2a8cadfd5562 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:44.977531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf649a5e-25cb-4d60-a26b-2a8cadfd5562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:44.995365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bf649a5e-25cb-4d60-a26b-2a8cadfd5562 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:44.996706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bf649a5e-25cb-4d60-a26b-2a8cadfd5562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:45.062244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 549/1108] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:45.329564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18562125-3221-4d88-8f8c-d2e5680aebc4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:45.428950 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18562125-3221-4d88-8f8c-d2e5680aebc4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:45.461925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d3beb-7177-4d01-822f-3203dd1f10e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:45.462536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d3beb-7177-4d01-822f-3203dd1f10e7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:45.462817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d3beb-7177-4d01-822f-3203dd1f10e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:45.463085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d3beb-7177-4d01-822f-3203dd1f10e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:45.464212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d3beb-7177-4d01-822f-3203dd1f10e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:45.465642 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d3beb-7177-4d01-822f-3203dd1f10e7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:45.467190 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 560/1109] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:45.473037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf649a5e-25cb-4d60-a26b-2a8cadfd5562 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:45.528899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:45.569586 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf649a5e-25cb-4d60-a26b-2a8cadfd5562 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:45.601341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ed2be-565f-485a-bb54-ed57ff99c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:45.601866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ed2be-565f-485a-bb54-ed57ff99c990 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:45.602142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ed2be-565f-485a-bb54-ed57ff99c990 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:45.602411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ed2be-565f-485a-bb54-ed57ff99c990 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:45.603037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ed2be-565f-485a-bb54-ed57ff99c990 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:45.604479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ed2be-565f-485a-bb54-ed57ff99c990 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:45.625727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 550/1110] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:51:45 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:51:47.553901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b8e75-6223-4f65-984b-30929e94837e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:47.554493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b8e75-6223-4f65-984b-30929e94837e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:47.554493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b8e75-6223-4f65-984b-30929e94837e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:47.555790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 561/1111] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:47.561094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9943b0-8a9a-47e6-8bc4-50b263be57ee None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:47.561785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9943b0-8a9a-47e6-8bc4-50b263be57ee None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:47.561884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9943b0-8a9a-47e6-8bc4-50b263be57ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:47.579924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ef9943b0-8a9a-47e6-8bc4-50b263be57ee None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:47.581135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ef9943b0-8a9a-47e6-8bc4-50b263be57ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:47.645510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 551/1112] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:47.913958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b8e75-6223-4f65-984b-30929e94837e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:47.964577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:48.014319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b8e75-6223-4f65-984b-30929e94837e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:48.046759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acd38d-de28-4309-bd70-f3f81eba8d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:48.047364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acd38d-de28-4309-bd70-f3f81eba8d00 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:48.047642 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acd38d-de28-4309-bd70-f3f81eba8d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:48.047913 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acd38d-de28-4309-bd70-f3f81eba8d00 None admin] PATH_INFO: `/v3/roles/fc1186ecd0f74b2fad8155d528f268f9` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:48.049320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acd38d-de28-4309-bd70-f3f81eba8d00 None admin] RBAC: Authorizing `identity:get_role(role_id=fc1186ecd0f74b2fad8155d528f268f9)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:48.050684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acd38d-de28-4309-bd70-f3f81eba8d00 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:48.052546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 562/1113] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:51:47 2022] GET /identity/v3/roles/fc1186ecd0f74b2fad8155d528f268f9 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:48.058322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9943b0-8a9a-47e6-8bc4-50b263be57ee None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:48.103438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:48.156356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9943b0-8a9a-47e6-8bc4-50b263be57ee None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:48.190012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbe50e-626d-431e-b2e7-8d83ce94340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:48.190630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbe50e-626d-431e-b2e7-8d83ce94340e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:48.190912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbe50e-626d-431e-b2e7-8d83ce94340e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:48.191189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbe50e-626d-431e-b2e7-8d83ce94340e None admin] PATH_INFO: `/v3/users/e3989d6dced74d4db2f58e8f8bbeaa0d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:48.204237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbe50e-626d-431e-b2e7-8d83ce94340e None admin] RBAC: Authorizing `identity:get_user(user_id=e3989d6dced74d4db2f58e8f8bbeaa0d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:48.205670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbe50e-626d-431e-b2e7-8d83ce94340e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:48.207303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 552/1114] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:51:48 2022] GET /identity/v3/users/e3989d6dced74d4db2f58e8f8bbeaa0d => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:48.213077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acd38d-de28-4309-bd70-f3f81eba8d00 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:48.311345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acd38d-de28-4309-bd70-f3f81eba8d00 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:48.328330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:48.344971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ebbe6-995e-40b1-bec9-eea780dd4fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:48.345558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ebbe6-995e-40b1-bec9-eea780dd4fcf None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:48.345836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ebbe6-995e-40b1-bec9-eea780dd4fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:48.346106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ebbe6-995e-40b1-bec9-eea780dd4fcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:48.412540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ebbe6-995e-40b1-bec9-eea780dd4fcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:48.413992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ebbe6-995e-40b1-bec9-eea780dd4fcf None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:48.507556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 563/1115] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:51:48.513657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbe50e-626d-431e-b2e7-8d83ce94340e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:48.610585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbe50e-626d-431e-b2e7-8d83ce94340e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:48.612114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:48.645684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a83de5-1901-4144-8a9a-056b946bac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:48.648517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a83de5-1901-4144-8a9a-056b946bac26 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:48.648517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a83de5-1901-4144-8a9a-056b946bac26 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:48.648517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a83de5-1901-4144-8a9a-056b946bac26 None admin] PATH_INFO: `/v3/projects/8de61c12e65748cca7cd676655bfee3a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:48.648517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a83de5-1901-4144-8a9a-056b946bac26 None admin] RBAC: Authorizing `identity:get_project(project_id=8de61c12e65748cca7cd676655bfee3a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:48.649808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a83de5-1901-4144-8a9a-056b946bac26 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:48.651458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 553/1116] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:51:48 2022] GET /identity/v3/projects/8de61c12e65748cca7cd676655bfee3a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:48.657661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ebbe6-995e-40b1-bec9-eea780dd4fcf None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:48.676950 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:48.758568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ebbe6-995e-40b1-bec9-eea780dd4fcf None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:48.792485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b07c65-f00b-4d02-9069-ddf49ebec0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:48.793080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b07c65-f00b-4d02-9069-ddf49ebec0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:48.793357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b07c65-f00b-4d02-9069-ddf49ebec0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:48.793627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b07c65-f00b-4d02-9069-ddf49ebec0c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:48.794344 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b07c65-f00b-4d02-9069-ddf49ebec0c2 None admin] RBAC: Adding query filter params (role.id=fc1186ecd0f74b2fad8155d528f268f9, scope.project.id=8de61c12e65748cca7cd676655bfee3a, user.id=e3989d6dced74d4db2f58e8f8bbeaa0d) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:48.794652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b07c65-f00b-4d02-9069-ddf49ebec0c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:48.796130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b07c65-f00b-4d02-9069-ddf49ebec0c2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:48.801436 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 564/1117] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:51:48 2022] GET /identity/v3/role_assignments?user.id=e3989d6dced74d4db2f58e8f8bbeaa0d&scope.project.id=8de61c12e65748cca7cd676655bfee3a&role.id=fc1186ecd0f74b2fad8155d528f268f9 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:50.667698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc38970-ac5c-4a2a-8785-971918bccfdf None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:50.668299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc38970-ac5c-4a2a-8785-971918bccfdf None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:50.668299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc38970-ac5c-4a2a-8785-971918bccfdf None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:50.669607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 554/1118] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:50.674732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de403625-5e3b-4e48-9e6f-e0f96ac18261 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:50.675163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de403625-5e3b-4e48-9e6f-e0f96ac18261 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:50.675266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de403625-5e3b-4e48-9e6f-e0f96ac18261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:50.694549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-de403625-5e3b-4e48-9e6f-e0f96ac18261 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:50.695787 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-de403625-5e3b-4e48-9e6f-e0f96ac18261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:50.761213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 565/1119] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:51.030902 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc38970-ac5c-4a2a-8785-971918bccfdf None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:51.131842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc38970-ac5c-4a2a-8785-971918bccfdf None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:51.164909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8a873-1440-48a0-bea0-9ec9dedc568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:51.165515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8a873-1440-48a0-bea0-9ec9dedc568b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:51.165797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8a873-1440-48a0-bea0-9ec9dedc568b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:51.166070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8a873-1440-48a0-bea0-9ec9dedc568b None admin] PATH_INFO: `/v3/roles/fc1186ecd0f74b2fad8155d528f268f9` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:51.167446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8a873-1440-48a0-bea0-9ec9dedc568b None admin] RBAC: Authorizing `identity:get_role(role_id=fc1186ecd0f74b2fad8155d528f268f9)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:51.168923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8a873-1440-48a0-bea0-9ec9dedc568b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:51.170232 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 555/1120] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:51:51 2022] GET /identity/v3/roles/fc1186ecd0f74b2fad8155d528f268f9 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:51.176187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de403625-5e3b-4e48-9e6f-e0f96ac18261 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:51.190675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:51.273564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de403625-5e3b-4e48-9e6f-e0f96ac18261 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:51.307254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac2961-1fbb-4d94-834f-610b6628adea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:51.307828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac2961-1fbb-4d94-834f-610b6628adea None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:51.308135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac2961-1fbb-4d94-834f-610b6628adea None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:51.308410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac2961-1fbb-4d94-834f-610b6628adea None admin] PATH_INFO: `/v3/users/e3989d6dced74d4db2f58e8f8bbeaa0d` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:51.309530 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac2961-1fbb-4d94-834f-610b6628adea None admin] RBAC: Authorizing `identity:get_user(user_id=e3989d6dced74d4db2f58e8f8bbeaa0d)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:51.311002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac2961-1fbb-4d94-834f-610b6628adea None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:51.312873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 566/1121] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:51:51 2022] GET /identity/v3/users/e3989d6dced74d4db2f58e8f8bbeaa0d => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:51.318687 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8a873-1440-48a0-bea0-9ec9dedc568b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:51.357205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:51.415202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8a873-1440-48a0-bea0-9ec9dedc568b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:51.448350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e074769-d9e1-4242-8f3a-fe929452af99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:51.448959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074769-d9e1-4242-8f3a-fe929452af99 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:51.449234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074769-d9e1-4242-8f3a-fe929452af99 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:51.449502 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074769-d9e1-4242-8f3a-fe929452af99 None admin] PATH_INFO: `/v3/projects/8de61c12e65748cca7cd676655bfee3a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:51.450594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e074769-d9e1-4242-8f3a-fe929452af99 None admin] RBAC: Authorizing `identity:get_project(project_id=8de61c12e65748cca7cd676655bfee3a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:51.451999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e074769-d9e1-4242-8f3a-fe929452af99 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:51.453631 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 556/1122] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:51:51 2022] GET /identity/v3/projects/8de61c12e65748cca7cd676655bfee3a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:51.460021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac2961-1fbb-4d94-834f-610b6628adea None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:51.539204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:51.558315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac2961-1fbb-4d94-834f-610b6628adea None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:51.591488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df3cc8-6aa2-4704-8b2e-a8aa656eaa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:51.592152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df3cc8-6aa2-4704-8b2e-a8aa656eaa7f None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:51.592432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df3cc8-6aa2-4704-8b2e-a8aa656eaa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:51.592703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df3cc8-6aa2-4704-8b2e-a8aa656eaa7f None admin] PATH_INFO: `/v3/projects/8de61c12e65748cca7cd676655bfee3a/users/e3989d6dced74d4db2f58e8f8bbeaa0d/roles/fc1186ecd0f74b2fad8155d528f268f9` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:51.594813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df3cc8-6aa2-4704-8b2e-a8aa656eaa7f None admin] RBAC: Authorizing `identity:create_grant(project_id=8de61c12e65748cca7cd676655bfee3a, user_id=e3989d6dced74d4db2f58e8f8bbeaa0d, role_id=fc1186ecd0f74b2fad8155d528f268f9)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:51.596263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df3cc8-6aa2-4704-8b2e-a8aa656eaa7f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:51.609133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 567/1123] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:51:51 2022] PUT /identity/v3/projects/8de61c12e65748cca7cd676655bfee3a/users/e3989d6dced74d4db2f58e8f8bbeaa0d/roles/fc1186ecd0f74b2fad8155d528f268f9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:51:53.485284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b76e7-f3f6-465a-ba45-07f19ff47703 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:53.485844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b76e7-f3f6-465a-ba45-07f19ff47703 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:53.485844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b76e7-f3f6-465a-ba45-07f19ff47703 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:53.487175 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 557/1124] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:53.493232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418dba2-8663-48b9-87b1-d85f0d337928 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:53.493717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418dba2-8663-48b9-87b1-d85f0d337928 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:53.493816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418dba2-8663-48b9-87b1-d85f0d337928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:53.511438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5418dba2-8663-48b9-87b1-d85f0d337928 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:53.512650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5418dba2-8663-48b9-87b1-d85f0d337928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:53.578444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 568/1125] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:53.845342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b76e7-f3f6-465a-ba45-07f19ff47703 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:53.947022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b76e7-f3f6-465a-ba45-07f19ff47703 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:53.954110 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:53.982619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2556719-acd2-41b1-b5af-1df6f779e8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:53.982918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2556719-acd2-41b1-b5af-1df6f779e8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:53.983113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2556719-acd2-41b1-b5af-1df6f779e8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:53.983666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2556719-acd2-41b1-b5af-1df6f779e8a6 None admin] PATH_INFO: `/v3/roles/fc1186ecd0f74b2fad8155d528f268f9` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:53.985236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2556719-acd2-41b1-b5af-1df6f779e8a6 None admin] RBAC: Authorizing `identity:get_role(role_id=fc1186ecd0f74b2fad8155d528f268f9)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:53.986530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2556719-acd2-41b1-b5af-1df6f779e8a6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:53.987865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 558/1126] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:51:53 2022] GET /identity/v3/roles/fc1186ecd0f74b2fad8155d528f268f9 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:53.993669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418dba2-8663-48b9-87b1-d85f0d337928 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:54.059467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:54.091141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418dba2-8663-48b9-87b1-d85f0d337928 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:54.123044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ccce3-34e3-4ff0-ae0b-5ca0fdbf7ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:54.123652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ccce3-34e3-4ff0-ae0b-5ca0fdbf7ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:54.123945 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ccce3-34e3-4ff0-ae0b-5ca0fdbf7ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:54.124233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ccce3-34e3-4ff0-ae0b-5ca0fdbf7ad4 None admin] PATH_INFO: `/v3/users/e3989d6dced74d4db2f58e8f8bbeaa0d` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:54.125331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ccce3-34e3-4ff0-ae0b-5ca0fdbf7ad4 None admin] RBAC: Authorizing `identity:get_user(user_id=e3989d6dced74d4db2f58e8f8bbeaa0d)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:54.126753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ccce3-34e3-4ff0-ae0b-5ca0fdbf7ad4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:54.128349 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 569/1127] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:51:53 2022] GET /identity/v3/users/e3989d6dced74d4db2f58e8f8bbeaa0d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:54.134715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2556719-acd2-41b1-b5af-1df6f779e8a6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:54.232447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2556719-acd2-41b1-b5af-1df6f779e8a6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:54.265649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d9bf2-8821-4d4f-a054-bbf8609840b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:54.266245 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d9bf2-8821-4d4f-a054-bbf8609840b4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:54.266527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d9bf2-8821-4d4f-a054-bbf8609840b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:54.266797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d9bf2-8821-4d4f-a054-bbf8609840b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:54.307763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:54.333347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d9bf2-8821-4d4f-a054-bbf8609840b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:54.335290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d9bf2-8821-4d4f-a054-bbf8609840b4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:54.429471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 559/1128] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:51:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:51:54.435662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ccce3-34e3-4ff0-ae0b-5ca0fdbf7ad4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:54.531120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ccce3-34e3-4ff0-ae0b-5ca0fdbf7ad4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:54.563811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff96f8-c518-472c-b008-7d464c213c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:54.564431 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff96f8-c518-472c-b008-7d464c213c7b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:54.564707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff96f8-c518-472c-b008-7d464c213c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:54.564975 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff96f8-c518-472c-b008-7d464c213c7b None admin] PATH_INFO: `/v3/projects/8de61c12e65748cca7cd676655bfee3a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:54.566063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff96f8-c518-472c-b008-7d464c213c7b None admin] RBAC: Authorizing `identity:get_project(project_id=8de61c12e65748cca7cd676655bfee3a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:54.567479 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff96f8-c518-472c-b008-7d464c213c7b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:54.569686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 570/1129] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:51:54 2022] GET /identity/v3/projects/8de61c12e65748cca7cd676655bfee3a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:54.575674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d9bf2-8821-4d4f-a054-bbf8609840b4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:54.647102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:54.671133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d9bf2-8821-4d4f-a054-bbf8609840b4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:54.705395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69340d-bb60-417c-b678-12d71851c2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:54.705975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69340d-bb60-417c-b678-12d71851c2cb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:54.706250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69340d-bb60-417c-b678-12d71851c2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:54.706516 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69340d-bb60-417c-b678-12d71851c2cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:54.707223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69340d-bb60-417c-b678-12d71851c2cb None admin] RBAC: Adding query filter params (role.id=fc1186ecd0f74b2fad8155d528f268f9, scope.project.id=8de61c12e65748cca7cd676655bfee3a, user.id=e3989d6dced74d4db2f58e8f8bbeaa0d) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:51:54.707526 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69340d-bb60-417c-b678-12d71851c2cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:54.708996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69340d-bb60-417c-b678-12d71851c2cb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:54.714510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 560/1130] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:51:54 2022] GET /identity/v3/role_assignments?user.id=e3989d6dced74d4db2f58e8f8bbeaa0d&scope.project.id=8de61c12e65748cca7cd676655bfee3a&role.id=fc1186ecd0f74b2fad8155d528f268f9 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:56.609412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f539a-ff2e-4f31-b6ea-68c468d0ddc0 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:56.609968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f539a-ff2e-4f31-b6ea-68c468d0ddc0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:56.609968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f539a-ff2e-4f31-b6ea-68c468d0ddc0 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:56.611279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 571/1131] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:56.616592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbae00-5407-4c87-8476-5c9cc944c548 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:56.617029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbae00-5407-4c87-8476-5c9cc944c548 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:56.617122 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbae00-5407-4c87-8476-5c9cc944c548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:56.635112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c8bbae00-5407-4c87-8476-5c9cc944c548 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:56.636332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c8bbae00-5407-4c87-8476-5c9cc944c548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:56.701687 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 561/1132] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:56.965866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f539a-ff2e-4f31-b6ea-68c468d0ddc0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:56.974696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:57.067350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f539a-ff2e-4f31-b6ea-68c468d0ddc0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:57.100879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0471d-2c03-40ad-ae41-799296d4d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:57.101489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0471d-2c03-40ad-ae41-799296d4d8fc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:57.101764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0471d-2c03-40ad-ae41-799296d4d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:57.102031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0471d-2c03-40ad-ae41-799296d4d8fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:57.103113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f0471d-2c03-40ad-ae41-799296d4d8fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:57.104569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f0471d-2c03-40ad-ae41-799296d4d8fc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:57.106122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 572/1133] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:57.111958 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbae00-5407-4c87-8476-5c9cc944c548 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:57.176035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:57.208755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbae00-5407-4c87-8476-5c9cc944c548 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:57.242127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e121f-0146-445d-80d1-525fe8b3023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:57.242662 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e121f-0146-445d-80d1-525fe8b3023c None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:57.242942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e121f-0146-445d-80d1-525fe8b3023c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:57.243214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e121f-0146-445d-80d1-525fe8b3023c None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:57.243860 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e121f-0146-445d-80d1-525fe8b3023c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:57.245314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e121f-0146-445d-80d1-525fe8b3023c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:57.272792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 562/1134] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:51:57 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:51:59.210625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb192-47f3-47ec-b8b0-d4ce59d7e8b8 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:59.211131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb192-47f3-47ec-b8b0-d4ce59d7e8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:59.211251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb192-47f3-47ec-b8b0-d4ce59d7e8b8 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:59.212541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 573/1135] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:59.217960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d6707-07f1-4ffa-a2c6-bb7302d46d1d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:59.218388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d6707-07f1-4ffa-a2c6-bb7302d46d1d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:59.218485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d6707-07f1-4ffa-a2c6-bb7302d46d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:59.236616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e49d6707-07f1-4ffa-a2c6-bb7302d46d1d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:59.237808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e49d6707-07f1-4ffa-a2c6-bb7302d46d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:59.304704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 563/1136] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:51:59.576340 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdb192-47f3-47ec-b8b0-d4ce59d7e8b8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:59.650199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:59.676212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdb192-47f3-47ec-b8b0-d4ce59d7e8b8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:59.709663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbd029-8b61-4909-a123-449436efcd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:59.710281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbd029-8b61-4909-a123-449436efcd5a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:59.710658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbd029-8b61-4909-a123-449436efcd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:59.710932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbd029-8b61-4909-a123-449436efcd5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:59.712038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbd029-8b61-4909-a123-449436efcd5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:59.713499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbd029-8b61-4909-a123-449436efcd5a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:59.715064 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 574/1137] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:51:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:51:59.720802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d6707-07f1-4ffa-a2c6-bb7302d46d1d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:59.772669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:59.823540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d6707-07f1-4ffa-a2c6-bb7302d46d1d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:59.857914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36a70e-4b8c-44b0-9f02-09122b4bed55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:59.858439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36a70e-4b8c-44b0-9f02-09122b4bed55 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:59.858717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36a70e-4b8c-44b0-9f02-09122b4bed55 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:59.858989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36a70e-4b8c-44b0-9f02-09122b4bed55 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:59.859639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36a70e-4b8c-44b0-9f02-09122b4bed55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:59.861108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36a70e-4b8c-44b0-9f02-09122b4bed55 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:59.882165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 564/1138] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:51:59 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:52:01.809475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bec39-2c3f-4342-a69f-42b5e3aa3bbb None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:01.809985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bec39-2c3f-4342-a69f-42b5e3aa3bbb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:01.810080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bec39-2c3f-4342-a69f-42b5e3aa3bbb None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:01.811444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 575/1139] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:01.816904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac6cb1-9cad-425d-98e6-c26474549efc None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:01.817307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac6cb1-9cad-425d-98e6-c26474549efc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:01.817455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac6cb1-9cad-425d-98e6-c26474549efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:01.836263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ddac6cb1-9cad-425d-98e6-c26474549efc None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:01.837443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ddac6cb1-9cad-425d-98e6-c26474549efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:01.901021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 565/1140] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:52:02.171007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bec39-2c3f-4342-a69f-42b5e3aa3bbb None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:02.271133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bec39-2c3f-4342-a69f-42b5e3aa3bbb None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:02.304171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dede04-2c2e-4e23-be1e-f30ecec15306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:02.304777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dede04-2c2e-4e23-be1e-f30ecec15306 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:02.305055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dede04-2c2e-4e23-be1e-f30ecec15306 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:02.305327 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dede04-2c2e-4e23-be1e-f30ecec15306 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:02.315538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dede04-2c2e-4e23-be1e-f30ecec15306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-58dede04-2c2e-4e23-be1e-f30ecec15306 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:52:02.320759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:52:02.321926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:52:02.323021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 576/1141] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:52:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:52:02.327808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac6cb1-9cad-425d-98e6-c26474549efc None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:02.361810 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:02.425089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac6cb1-9cad-425d-98e6-c26474549efc None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:02.459216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc48441-de48-4c40-9a98-06c1964008bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:02.459840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48441-de48-4c40-9a98-06c1964008bb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:02.460150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48441-de48-4c40-9a98-06c1964008bb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:02.460428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48441-de48-4c40-9a98-06c1964008bb None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:02.461091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc48441-de48-4c40-9a98-06c1964008bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:52:02.461396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc48441-de48-4c40-9a98-06c1964008bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:02.462762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc48441-de48-4c40-9a98-06c1964008bb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:02.470566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 566/1142] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:52:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:02.476894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dede04-2c2e-4e23-be1e-f30ecec15306 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:02.480334 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:02.580151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dede04-2c2e-4e23-be1e-f30ecec15306 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:02.613474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8858edc-1ea4-4515-b7e2-43fd1da63ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:02.614131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8858edc-1ea4-4515-b7e2-43fd1da63ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:02.614412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8858edc-1ea4-4515-b7e2-43fd1da63ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:02.614710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8858edc-1ea4-4515-b7e2-43fd1da63ad2 None admin] PATH_INFO: `/v3/users/1c26a2f37dd94b5a8aeb217dc32bce36` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:02.628086 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8858edc-1ea4-4515-b7e2-43fd1da63ad2 None admin] RBAC: Authorizing `identity:get_user(user_id=1c26a2f37dd94b5a8aeb217dc32bce36)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:02.629535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8858edc-1ea4-4515-b7e2-43fd1da63ad2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:02.631133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 577/1143] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:52:02 2022] GET /identity/v3/users/1c26a2f37dd94b5a8aeb217dc32bce36 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:02.637097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc48441-de48-4c40-9a98-06c1964008bb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:02.731406 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:02.735442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc48441-de48-4c40-9a98-06c1964008bb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:02.767808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52343caa-68fe-4233-9fb8-1911803d9416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:02.768447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52343caa-68fe-4233-9fb8-1911803d9416 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:02.768730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52343caa-68fe-4233-9fb8-1911803d9416 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:02.769044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52343caa-68fe-4233-9fb8-1911803d9416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:02.841320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52343caa-68fe-4233-9fb8-1911803d9416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:02.842879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52343caa-68fe-4233-9fb8-1911803d9416 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:02.934794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:02.939927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 567/1144] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:52:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:52:02.945721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8858edc-1ea4-4515-b7e2-43fd1da63ad2 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:03.002864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:03.043751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8858edc-1ea4-4515-b7e2-43fd1da63ad2 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:03.077534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de06374-34f7-4f5b-bc87-3775041efba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:03.078022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06374-34f7-4f5b-bc87-3775041efba2 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:03.078303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06374-34f7-4f5b-bc87-3775041efba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:03.078579 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06374-34f7-4f5b-bc87-3775041efba2 None admin] PATH_INFO: `/v3/projects/efe6cd3820bb4a2bbf726d13fdd0e240` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:03.079702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de06374-34f7-4f5b-bc87-3775041efba2 None admin] RBAC: Authorizing `identity:get_project(project_id=efe6cd3820bb4a2bbf726d13fdd0e240)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:03.081180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de06374-34f7-4f5b-bc87-3775041efba2 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:03.082818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 578/1145] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:52:02 2022] GET /identity/v3/projects/efe6cd3820bb4a2bbf726d13fdd0e240 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:03.089528 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52343caa-68fe-4233-9fb8-1911803d9416 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:03.185930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52343caa-68fe-4233-9fb8-1911803d9416 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:03.219723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd681174-f135-446c-a2b3-f4f8089b6875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:03.220343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd681174-f135-446c-a2b3-f4f8089b6875 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:03.220622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd681174-f135-446c-a2b3-f4f8089b6875 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:03.220892 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd681174-f135-446c-a2b3-f4f8089b6875 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:03.221609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd681174-f135-446c-a2b3-f4f8089b6875 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=efe6cd3820bb4a2bbf726d13fdd0e240, user.id=1c26a2f37dd94b5a8aeb217dc32bce36) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:52:03.221919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd681174-f135-446c-a2b3-f4f8089b6875 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:03.223351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd681174-f135-446c-a2b3-f4f8089b6875 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:03.228597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 568/1146] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:52:03 2022] GET /identity/v3/role_assignments?user.id=1c26a2f37dd94b5a8aeb217dc32bce36&scope.project.id=efe6cd3820bb4a2bbf726d13fdd0e240&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:05.105255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073e437-47c0-48c5-9fab-a79da0370932 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:05.105776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073e437-47c0-48c5-9fab-a79da0370932 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:05.105776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073e437-47c0-48c5-9fab-a79da0370932 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:05.107076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 579/1147] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:05.112131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085af53-bc3b-4197-a2ed-b8e0ad7618ae None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:05.112554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085af53-bc3b-4197-a2ed-b8e0ad7618ae None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:05.112685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085af53-bc3b-4197-a2ed-b8e0ad7618ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:05.130322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7085af53-bc3b-4197-a2ed-b8e0ad7618ae None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:05.131507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7085af53-bc3b-4197-a2ed-b8e0ad7618ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:05.186670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:05.197822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 569/1148] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:52:05.463035 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073e437-47c0-48c5-9fab-a79da0370932 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:05.561118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073e437-47c0-48c5-9fab-a79da0370932 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:05.587885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:05.596019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7a33c-944d-41b4-a5e8-52fb6a0742be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:05.596656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7a33c-944d-41b4-a5e8-52fb6a0742be None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:05.596939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7a33c-944d-41b4-a5e8-52fb6a0742be None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:05.597215 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7a33c-944d-41b4-a5e8-52fb6a0742be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:05.607029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7a33c-944d-41b4-a5e8-52fb6a0742be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-70b7a33c-944d-41b4-a5e8-52fb6a0742be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:52:05.611635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:52:05.612775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:52:05.613641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 580/1149] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:52:05.618658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085af53-bc3b-4197-a2ed-b8e0ad7618ae None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:05.716102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085af53-bc3b-4197-a2ed-b8e0ad7618ae None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:05.749172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e7d65-6b09-4b1d-851e-fb9f172a06d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:05.749776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e7d65-6b09-4b1d-851e-fb9f172a06d7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:05.750059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e7d65-6b09-4b1d-851e-fb9f172a06d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:05.750334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e7d65-6b09-4b1d-851e-fb9f172a06d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:05.750999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e7d65-6b09-4b1d-851e-fb9f172a06d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:52:05.751311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e7d65-6b09-4b1d-851e-fb9f172a06d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:05.752710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e7d65-6b09-4b1d-851e-fb9f172a06d7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:05.761602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 570/1150] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:52:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:05.767480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7a33c-944d-41b4-a5e8-52fb6a0742be None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:05.863757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7a33c-944d-41b4-a5e8-52fb6a0742be None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:05.896919 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167c935-bdcd-4a07-9561-fe0c2f17bc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:05.897527 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167c935-bdcd-4a07-9561-fe0c2f17bc07 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:05.897827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167c935-bdcd-4a07-9561-fe0c2f17bc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:05.898099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167c935-bdcd-4a07-9561-fe0c2f17bc07 None admin] PATH_INFO: `/v3/users/1c26a2f37dd94b5a8aeb217dc32bce36` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:05.899197 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167c935-bdcd-4a07-9561-fe0c2f17bc07 None admin] RBAC: Authorizing `identity:get_user(user_id=1c26a2f37dd94b5a8aeb217dc32bce36)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:05.900655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167c935-bdcd-4a07-9561-fe0c2f17bc07 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:05.902218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 581/1151] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:52:05 2022] GET /identity/v3/users/1c26a2f37dd94b5a8aeb217dc32bce36 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:05.908064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e7d65-6b09-4b1d-851e-fb9f172a06d7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:05.963670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:06.008827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e7d65-6b09-4b1d-851e-fb9f172a06d7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:06.041700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367ca48-7997-4a4c-b0b4-a80ee37b15b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:06.042319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367ca48-7997-4a4c-b0b4-a80ee37b15b9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:06.042600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367ca48-7997-4a4c-b0b4-a80ee37b15b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:06.042875 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367ca48-7997-4a4c-b0b4-a80ee37b15b9 None admin] PATH_INFO: `/v3/projects/efe6cd3820bb4a2bbf726d13fdd0e240` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:06.043979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367ca48-7997-4a4c-b0b4-a80ee37b15b9 None admin] RBAC: Authorizing `identity:get_project(project_id=efe6cd3820bb4a2bbf726d13fdd0e240)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:06.045439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367ca48-7997-4a4c-b0b4-a80ee37b15b9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:06.047063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 571/1152] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:52:05 2022] GET /identity/v3/projects/efe6cd3820bb4a2bbf726d13fdd0e240 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:06.052985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167c935-bdcd-4a07-9561-fe0c2f17bc07 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:06.108105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:06.151602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167c935-bdcd-4a07-9561-fe0c2f17bc07 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:06.185729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177dade-94eb-436f-a161-134ab910ed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:06.186369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177dade-94eb-436f-a161-134ab910ed59 None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:06.186650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177dade-94eb-436f-a161-134ab910ed59 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:06.186924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177dade-94eb-436f-a161-134ab910ed59 None admin] PATH_INFO: `/v3/projects/efe6cd3820bb4a2bbf726d13fdd0e240/users/1c26a2f37dd94b5a8aeb217dc32bce36/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:06.188846 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177dade-94eb-436f-a161-134ab910ed59 None admin] RBAC: Authorizing `identity:create_grant(project_id=efe6cd3820bb4a2bbf726d13fdd0e240, user_id=1c26a2f37dd94b5a8aeb217dc32bce36, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:06.190554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177dade-94eb-436f-a161-134ab910ed59 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:06.203144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 582/1153] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:52:06 2022] PUT /identity/v3/projects/efe6cd3820bb4a2bbf726d13fdd0e240/users/1c26a2f37dd94b5a8aeb217dc32bce36/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:52:08.068868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee246e2-3b21-42c1-b13a-1d02d3b2981b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:08.069427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee246e2-3b21-42c1-b13a-1d02d3b2981b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:08.069427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee246e2-3b21-42c1-b13a-1d02d3b2981b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:08.070687 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 572/1154] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:08.075769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3d805-2e30-42ad-93bc-f43acde1959d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:08.076161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3d805-2e30-42ad-93bc-f43acde1959d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:08.076358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3d805-2e30-42ad-93bc-f43acde1959d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:08.095023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-92f3d805-2e30-42ad-93bc-f43acde1959d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:08.096240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-92f3d805-2e30-42ad-93bc-f43acde1959d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:08.160968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 583/1155] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:52:08.434262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee246e2-3b21-42c1-b13a-1d02d3b2981b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:08.533948 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee246e2-3b21-42c1-b13a-1d02d3b2981b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:08.556186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:08.567988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdbec5-d2db-413f-add6-f47e22d6d705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:08.568626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdbec5-d2db-413f-add6-f47e22d6d705 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:08.568905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdbec5-d2db-413f-add6-f47e22d6d705 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:08.569172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdbec5-d2db-413f-add6-f47e22d6d705 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:08.579470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdbec5-d2db-413f-add6-f47e22d6d705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e9bdbec5-d2db-413f-add6-f47e22d6d705 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:52:08.584066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:52:08.585143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:52:08.586439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 573/1156] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:52:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:52:08.591160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3d805-2e30-42ad-93bc-f43acde1959d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:08.637496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:08.689688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3d805-2e30-42ad-93bc-f43acde1959d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:08.723007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87702d6-1cd5-45a1-9b95-154238c9afbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:08.723591 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87702d6-1cd5-45a1-9b95-154238c9afbe None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:08.723872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87702d6-1cd5-45a1-9b95-154238c9afbe None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:08.724173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87702d6-1cd5-45a1-9b95-154238c9afbe None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:08.724872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87702d6-1cd5-45a1-9b95-154238c9afbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:52:08.725174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87702d6-1cd5-45a1-9b95-154238c9afbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:08.727081 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87702d6-1cd5-45a1-9b95-154238c9afbe None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:08.735228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 584/1157] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:52:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:08.741025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdbec5-d2db-413f-add6-f47e22d6d705 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:08.838634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdbec5-d2db-413f-add6-f47e22d6d705 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:08.871222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3f2ca-72a3-47fc-b4c4-1ed492d4318a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:08.871842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3f2ca-72a3-47fc-b4c4-1ed492d4318a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:08.872153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3f2ca-72a3-47fc-b4c4-1ed492d4318a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:08.872431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3f2ca-72a3-47fc-b4c4-1ed492d4318a None admin] PATH_INFO: `/v3/users/1c26a2f37dd94b5a8aeb217dc32bce36` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:08.873545 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3f2ca-72a3-47fc-b4c4-1ed492d4318a None admin] RBAC: Authorizing `identity:get_user(user_id=1c26a2f37dd94b5a8aeb217dc32bce36)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:08.874970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3f2ca-72a3-47fc-b4c4-1ed492d4318a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:08.876603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 574/1158] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:52:08 2022] GET /identity/v3/users/1c26a2f37dd94b5a8aeb217dc32bce36 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:08.882517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87702d6-1cd5-45a1-9b95-154238c9afbe None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:08.981165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87702d6-1cd5-45a1-9b95-154238c9afbe None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:08.989609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:09.014199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb91ae5-7b81-437c-8c8d-0cd90bf4609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:09.014789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb91ae5-7b81-437c-8c8d-0cd90bf4609d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:09.015069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb91ae5-7b81-437c-8c8d-0cd90bf4609d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:09.015341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb91ae5-7b81-437c-8c8d-0cd90bf4609d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:09.087116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb91ae5-7b81-437c-8c8d-0cd90bf4609d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:09.088601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb91ae5-7b81-437c-8c8d-0cd90bf4609d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:09.183326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 585/1159] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:52:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:52:09.189385 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3f2ca-72a3-47fc-b4c4-1ed492d4318a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:09.239009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:09.289440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3f2ca-72a3-47fc-b4c4-1ed492d4318a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:09.322306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b5d9e-9bbc-4a38-98d0-0ee1f0e278e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:09.322922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b5d9e-9bbc-4a38-98d0-0ee1f0e278e4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:09.323205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b5d9e-9bbc-4a38-98d0-0ee1f0e278e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:09.323482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b5d9e-9bbc-4a38-98d0-0ee1f0e278e4 None admin] PATH_INFO: `/v3/projects/efe6cd3820bb4a2bbf726d13fdd0e240` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:09.324609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b5d9e-9bbc-4a38-98d0-0ee1f0e278e4 None admin] RBAC: Authorizing `identity:get_project(project_id=efe6cd3820bb4a2bbf726d13fdd0e240)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:09.326041 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b5d9e-9bbc-4a38-98d0-0ee1f0e278e4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:09.327664 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 575/1160] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:52:09 2022] GET /identity/v3/projects/efe6cd3820bb4a2bbf726d13fdd0e240 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:09.333969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb91ae5-7b81-437c-8c8d-0cd90bf4609d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:09.344497 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:09.431288 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb91ae5-7b81-437c-8c8d-0cd90bf4609d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:09.464596 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10467cb-f789-41ea-a47d-aa63e0f50b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:09.465190 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10467cb-f789-41ea-a47d-aa63e0f50b01 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:09.465473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10467cb-f789-41ea-a47d-aa63e0f50b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:09.465746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10467cb-f789-41ea-a47d-aa63e0f50b01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:09.466467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10467cb-f789-41ea-a47d-aa63e0f50b01 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=efe6cd3820bb4a2bbf726d13fdd0e240, user.id=1c26a2f37dd94b5a8aeb217dc32bce36) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:52:09.466781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10467cb-f789-41ea-a47d-aa63e0f50b01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:09.468248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10467cb-f789-41ea-a47d-aa63e0f50b01 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:09.473841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 586/1161] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:52:09 2022] GET /identity/v3/role_assignments?user.id=1c26a2f37dd94b5a8aeb217dc32bce36&scope.project.id=efe6cd3820bb4a2bbf726d13fdd0e240&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:11.348405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b4df5-ffcb-4919-98e6-5dfc76cd9ed9 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:11.348943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b4df5-ffcb-4919-98e6-5dfc76cd9ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:11.348943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b4df5-ffcb-4919-98e6-5dfc76cd9ed9 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:11.350297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 576/1162] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:11.355909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb3a82-26e4-4f54-b033-dbf6f90c600b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:11.356353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb3a82-26e4-4f54-b033-dbf6f90c600b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:11.356493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb3a82-26e4-4f54-b033-dbf6f90c600b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:11.374940 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cdbb3a82-26e4-4f54-b033-dbf6f90c600b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:11.376160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cdbb3a82-26e4-4f54-b033-dbf6f90c600b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:11.440693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 587/1163] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:52:11.704022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b4df5-ffcb-4919-98e6-5dfc76cd9ed9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:11.803005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b4df5-ffcb-4919-98e6-5dfc76cd9ed9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:11.819937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:11.836957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454036c8-c6b4-40b1-8c77-6c6f944150ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:11.837561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454036c8-c6b4-40b1-8c77-6c6f944150ff None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:11.837838 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454036c8-c6b4-40b1-8c77-6c6f944150ff None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:11.838116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454036c8-c6b4-40b1-8c77-6c6f944150ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:11.902989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454036c8-c6b4-40b1-8c77-6c6f944150ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:11.904450 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454036c8-c6b4-40b1-8c77-6c6f944150ff None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:11.999535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 577/1164] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:52:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:52:12.005404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb3a82-26e4-4f54-b033-dbf6f90c600b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:12.007572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:12.103691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb3a82-26e4-4f54-b033-dbf6f90c600b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:12.136525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb8db3-2750-4387-b09d-cd5412bef39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:12.137147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb8db3-2750-4387-b09d-cd5412bef39c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:12.137427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb8db3-2750-4387-b09d-cd5412bef39c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:12.137701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb8db3-2750-4387-b09d-cd5412bef39c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:12.144246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fb8db3-2750-4387-b09d-cd5412bef39c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:12.145690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fb8db3-2750-4387-b09d-cd5412bef39c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-24fb8db3-2750-4387-b09d-cd5412bef39c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:52:12.157108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:52:12.158271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:52:12.159307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 588/1165] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Aug 24 09:52:12 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:52:12.164224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454036c8-c6b4-40b1-8c77-6c6f944150ff None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:12.183070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:12.261038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454036c8-c6b4-40b1-8c77-6c6f944150ff None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:12.293653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033fd19-ed1a-4b25-92ea-9cc48ea63fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:12.294248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033fd19-ed1a-4b25-92ea-9cc48ea63fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:12.294526 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033fd19-ed1a-4b25-92ea-9cc48ea63fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:12.294797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033fd19-ed1a-4b25-92ea-9cc48ea63fd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:12.295426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033fd19-ed1a-4b25-92ea-9cc48ea63fd7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:52:12.295727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033fd19-ed1a-4b25-92ea-9cc48ea63fd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:12.297113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033fd19-ed1a-4b25-92ea-9cc48ea63fd7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:12.303968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 578/1166] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Aug 24 09:52:12 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:14.170556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091e1d6-0f26-4d29-8842-49ab3a6835d9 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:14.171056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091e1d6-0f26-4d29-8842-49ab3a6835d9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:14.171056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091e1d6-0f26-4d29-8842-49ab3a6835d9 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:14.172398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 589/1167] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:14.177732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f489af-cb78-42df-882a-66fce237154d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:14.178142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f489af-cb78-42df-882a-66fce237154d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:14.178276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f489af-cb78-42df-882a-66fce237154d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:14.196057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-85f489af-cb78-42df-882a-66fce237154d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:14.197324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-85f489af-cb78-42df-882a-66fce237154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:14.261439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 579/1168] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:52:14.525983 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091e1d6-0f26-4d29-8842-49ab3a6835d9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:14.586474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:14.626236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091e1d6-0f26-4d29-8842-49ab3a6835d9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:14.659593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4443656-ff02-4b81-94dc-c4207db2cf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:14.660218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443656-ff02-4b81-94dc-c4207db2cf4b None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:14.660505 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443656-ff02-4b81-94dc-c4207db2cf4b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:14.660777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443656-ff02-4b81-94dc-c4207db2cf4b None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:14.661443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4443656-ff02-4b81-94dc-c4207db2cf4b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:14.662816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4443656-ff02-4b81-94dc-c4207db2cf4b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:14.678307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 590/1169] 10.222.0.86 () {60 vars in 1138 bytes} [Wed Aug 24 09:52:14 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:52:16.594709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f62d81-3566-4d4a-85a7-347a052e2b40 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:16.595376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f62d81-3566-4d4a-85a7-347a052e2b40 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:16.595376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f62d81-3566-4d4a-85a7-347a052e2b40 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:16.596643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 580/1170] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:16.602312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd62b3-54d0-4b49-94f7-628d4ca0a2c9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:16.602709 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd62b3-54d0-4b49-94f7-628d4ca0a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:16.602860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd62b3-54d0-4b49-94f7-628d4ca0a2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:16.624791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d3bd62b3-54d0-4b49-94f7-628d4ca0a2c9 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:16.625966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d3bd62b3-54d0-4b49-94f7-628d4ca0a2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:16.691554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 591/1171] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:52:16.959746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f62d81-3566-4d4a-85a7-347a052e2b40 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:16.974823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:17.059705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f62d81-3566-4d4a-85a7-347a052e2b40 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:17.092396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fcb2a-5a8b-40a7-aef5-9d1185f36cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:17.093005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fcb2a-5a8b-40a7-aef5-9d1185f36cdf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:17.093283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fcb2a-5a8b-40a7-aef5-9d1185f36cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:17.093557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fcb2a-5a8b-40a7-aef5-9d1185f36cdf None admin] PATH_INFO: `/v3/domains/76cf9e9722da47e3846839b9cfb3eac8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:17.104699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fcb2a-5a8b-40a7-aef5-9d1185f36cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=76cf9e9722da47e3846839b9cfb3eac8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:17.106136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fcb2a-5a8b-40a7-aef5-9d1185f36cdf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:17.107739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 581/1172] 10.222.0.86 () {56 vars in 1183 bytes} [Wed Aug 24 09:52:16 2022] GET /identity/v3/domains/76cf9e9722da47e3846839b9cfb3eac8 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:17.113577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd62b3-54d0-4b49-94f7-628d4ca0a2c9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:17.163317 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:17.209414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd62b3-54d0-4b49-94f7-628d4ca0a2c9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:17.244450 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6464078-bc9e-4bc5-8853-0582e0b83a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:17.244989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6464078-bc9e-4bc5-8853-0582e0b83a66 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:17.245264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6464078-bc9e-4bc5-8853-0582e0b83a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:17.245532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6464078-bc9e-4bc5-8853-0582e0b83a66 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:17.246162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6464078-bc9e-4bc5-8853-0582e0b83a66 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:17.247578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6464078-bc9e-4bc5-8853-0582e0b83a66 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:17.272845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 592/1173] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:52:17 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:52:19.156255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0581-9820-4bf4-9933-18b1564c0b0b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:19.156761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0581-9820-4bf4-9933-18b1564c0b0b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:19.156761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0581-9820-4bf4-9933-18b1564c0b0b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:19.158210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 582/1174] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:19.163354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c36bc6-d40e-4301-9ad2-01be71f5373b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:19.163776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c36bc6-d40e-4301-9ad2-01be71f5373b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:19.163911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c36bc6-d40e-4301-9ad2-01be71f5373b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:19.182019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-26c36bc6-d40e-4301-9ad2-01be71f5373b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:19.183196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-26c36bc6-d40e-4301-9ad2-01be71f5373b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:19.248059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 593/1175] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:52:19.512366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade0581-9820-4bf4-9933-18b1564c0b0b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:19.582644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:19.632735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade0581-9820-4bf4-9933-18b1564c0b0b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:19.684580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c57cc-5b28-453a-b3f3-082e1790faba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:19.685393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c57cc-5b28-453a-b3f3-082e1790faba None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:19.685801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c57cc-5b28-453a-b3f3-082e1790faba None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:19.686138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c57cc-5b28-453a-b3f3-082e1790faba None admin] PATH_INFO: `/v3/domains/76cf9e9722da47e3846839b9cfb3eac8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:19.687263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c57cc-5b28-453a-b3f3-082e1790faba None admin] RBAC: Authorizing `identity:get_domain(domain_id=76cf9e9722da47e3846839b9cfb3eac8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:19.688723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c57cc-5b28-453a-b3f3-082e1790faba None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:19.690703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 583/1176] 10.222.0.86 () {56 vars in 1183 bytes} [Wed Aug 24 09:52:19 2022] GET /identity/v3/domains/76cf9e9722da47e3846839b9cfb3eac8 => generated 267 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:19.696800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c36bc6-d40e-4301-9ad2-01be71f5373b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:19.729456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:19.795878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c36bc6-d40e-4301-9ad2-01be71f5373b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:19.829026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab01b93-82a0-4de9-a911-217beeea04a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:19.829620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab01b93-82a0-4de9-a911-217beeea04a0 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:19.830046 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab01b93-82a0-4de9-a911-217beeea04a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:19.830171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab01b93-82a0-4de9-a911-217beeea04a0 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:19.830795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab01b93-82a0-4de9-a911-217beeea04a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:19.832250 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab01b93-82a0-4de9-a911-217beeea04a0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:19.853686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 594/1177] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:52:19 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:52:21.766953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f18e4-eaa6-4742-a942-bad660249502 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:21.767509 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f18e4-eaa6-4742-a942-bad660249502 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:21.767509 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f18e4-eaa6-4742-a942-bad660249502 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:21.769305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 584/1178] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:21.774283 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51863fd-b86e-4e79-8d0b-b48dfba63132 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:21.774674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51863fd-b86e-4e79-8d0b-b48dfba63132 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:21.774834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51863fd-b86e-4e79-8d0b-b48dfba63132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:21.792651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c51863fd-b86e-4e79-8d0b-b48dfba63132 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:21.793849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c51863fd-b86e-4e79-8d0b-b48dfba63132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:21.859195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 595/1179] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:52:22.130520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f18e4-eaa6-4742-a942-bad660249502 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:22.231160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f18e4-eaa6-4742-a942-bad660249502 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:22.266705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b4a34-7bc7-4f48-9fc2-3061dbd24d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:22.267324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b4a34-7bc7-4f48-9fc2-3061dbd24d0f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:22.267606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b4a34-7bc7-4f48-9fc2-3061dbd24d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:22.267904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b4a34-7bc7-4f48-9fc2-3061dbd24d0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:22.275282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:22.280098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b4a34-7bc7-4f48-9fc2-3061dbd24d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-668b4a34-7bc7-4f48-9fc2-3061dbd24d0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:52:22.284877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:52:22.286682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:52:22.287784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 585/1180] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:52:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:52:22.292682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51863fd-b86e-4e79-8d0b-b48dfba63132 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:22.313003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:22.391417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51863fd-b86e-4e79-8d0b-b48dfba63132 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:22.424583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d10f71-8331-4144-b875-43a2287eb14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:22.425176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d10f71-8331-4144-b875-43a2287eb14c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:22.425457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d10f71-8331-4144-b875-43a2287eb14c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:22.425732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d10f71-8331-4144-b875-43a2287eb14c None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:22.426393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d10f71-8331-4144-b875-43a2287eb14c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:52:22.426695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d10f71-8331-4144-b875-43a2287eb14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:22.428089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d10f71-8331-4144-b875-43a2287eb14c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:22.435848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 596/1181] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:52:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:52:22.441942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b4a34-7bc7-4f48-9fc2-3061dbd24d0f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:22.540288 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b4a34-7bc7-4f48-9fc2-3061dbd24d0f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:22.574040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec004e-2d1c-4600-b800-b9c65a0eaad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:22.574659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec004e-2d1c-4600-b800-b9c65a0eaad8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:22.574942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec004e-2d1c-4600-b800-b9c65a0eaad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:22.575218 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec004e-2d1c-4600-b800-b9c65a0eaad8 None admin] PATH_INFO: `/v3/users/1aa924e3cfe44a03bd04fd9796b5ffd0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:22.587881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec004e-2d1c-4600-b800-b9c65a0eaad8 None admin] RBAC: Authorizing `identity:get_user(user_id=1aa924e3cfe44a03bd04fd9796b5ffd0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:22.589877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec004e-2d1c-4600-b800-b9c65a0eaad8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:22.591467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 586/1182] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:52:22 2022] GET /identity/v3/users/1aa924e3cfe44a03bd04fd9796b5ffd0 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:22.597400 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d10f71-8331-4144-b875-43a2287eb14c None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:22.676390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:22.694201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d10f71-8331-4144-b875-43a2287eb14c None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:22.727181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed85cac-aeb5-4707-a3db-576e1aad82b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:22.727773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed85cac-aeb5-4707-a3db-576e1aad82b3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:22.728074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed85cac-aeb5-4707-a3db-576e1aad82b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:22.728357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed85cac-aeb5-4707-a3db-576e1aad82b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:22.795135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed85cac-aeb5-4707-a3db-576e1aad82b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:22.796619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed85cac-aeb5-4707-a3db-576e1aad82b3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:22.873766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:22.891963 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 597/1183] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:52:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:52:22.897908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec004e-2d1c-4600-b800-b9c65a0eaad8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:22.955048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:22.995883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec004e-2d1c-4600-b800-b9c65a0eaad8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:23.028700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5189ef-5156-4471-9099-1ee17f1c68be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:23.029313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5189ef-5156-4471-9099-1ee17f1c68be None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:23.034512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5189ef-5156-4471-9099-1ee17f1c68be None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:23.034512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5189ef-5156-4471-9099-1ee17f1c68be None admin] PATH_INFO: `/v3/projects/7f24a74e6d2d469cadea57d573f396bd` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:23.034512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5189ef-5156-4471-9099-1ee17f1c68be None admin] RBAC: Authorizing `identity:get_project(project_id=7f24a74e6d2d469cadea57d573f396bd)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:23.034512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5189ef-5156-4471-9099-1ee17f1c68be None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:23.034512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 587/1184] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:52:22 2022] GET /identity/v3/projects/7f24a74e6d2d469cadea57d573f396bd => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:23.041983 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed85cac-aeb5-4707-a3db-576e1aad82b3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:23.137602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed85cac-aeb5-4707-a3db-576e1aad82b3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:23.171454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3787b4-eedd-4b48-80c3-0912f9f2db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:23.172059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3787b4-eedd-4b48-80c3-0912f9f2db67 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:23.172345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3787b4-eedd-4b48-80c3-0912f9f2db67 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:23.172611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3787b4-eedd-4b48-80c3-0912f9f2db67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:23.173325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3787b4-eedd-4b48-80c3-0912f9f2db67 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=7f24a74e6d2d469cadea57d573f396bd, user.id=1aa924e3cfe44a03bd04fd9796b5ffd0) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:52:23.173627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3787b4-eedd-4b48-80c3-0912f9f2db67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:23.175051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3787b4-eedd-4b48-80c3-0912f9f2db67 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:23.180281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 598/1185] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:52:23 2022] GET /identity/v3/role_assignments?user.id=1aa924e3cfe44a03bd04fd9796b5ffd0&scope.project.id=7f24a74e6d2d469cadea57d573f396bd&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:25.019360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca61b1-2fac-419b-ac7c-90ae1227e20c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:25.019878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca61b1-2fac-419b-ac7c-90ae1227e20c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:25.019878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca61b1-2fac-419b-ac7c-90ae1227e20c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:25.021200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 588/1186] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:25.026364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be666542-d114-4187-9094-4474f63cd2fc None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:25.026750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be666542-d114-4187-9094-4474f63cd2fc None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:25.026906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be666542-d114-4187-9094-4474f63cd2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:25.044937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-be666542-d114-4187-9094-4474f63cd2fc None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:25.046105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-be666542-d114-4187-9094-4474f63cd2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:25.087152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:25.112266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 599/1187] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:52:25.377338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca61b1-2fac-419b-ac7c-90ae1227e20c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:25.476602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca61b1-2fac-419b-ac7c-90ae1227e20c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:25.501530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:25.510360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf68b8-d3d9-47c1-b40c-ef5b24ecf3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:25.510976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf68b8-d3d9-47c1-b40c-ef5b24ecf3ed None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:25.511257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf68b8-d3d9-47c1-b40c-ef5b24ecf3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:25.511528 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf68b8-d3d9-47c1-b40c-ef5b24ecf3ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:25.521426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf68b8-d3d9-47c1-b40c-ef5b24ecf3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4caf68b8-d3d9-47c1-b40c-ef5b24ecf3ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:52:25.526547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:52:25.527669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:52:25.528772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 589/1188] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:52:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:52:25.533724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be666542-d114-4187-9094-4474f63cd2fc None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:25.627836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be666542-d114-4187-9094-4474f63cd2fc None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:25.661577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2390e36-fe27-49af-9ab6-6b92e1e825a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:25.662173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390e36-fe27-49af-9ab6-6b92e1e825a8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:25.662458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390e36-fe27-49af-9ab6-6b92e1e825a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:25.662730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390e36-fe27-49af-9ab6-6b92e1e825a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:25.663391 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2390e36-fe27-49af-9ab6-6b92e1e825a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:52:25.663695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2390e36-fe27-49af-9ab6-6b92e1e825a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:25.665092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2390e36-fe27-49af-9ab6-6b92e1e825a8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:25.672954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 600/1189] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:52:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:25.678597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf68b8-d3d9-47c1-b40c-ef5b24ecf3ed None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:25.775775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf68b8-d3d9-47c1-b40c-ef5b24ecf3ed None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:25.809902 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e3a0e-c6cd-4916-872e-06deb1519453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:25.810506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3a0e-c6cd-4916-872e-06deb1519453 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:25.810782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3a0e-c6cd-4916-872e-06deb1519453 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:25.811051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3a0e-c6cd-4916-872e-06deb1519453 None admin] PATH_INFO: `/v3/users/1aa924e3cfe44a03bd04fd9796b5ffd0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:25.812170 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e3a0e-c6cd-4916-872e-06deb1519453 None admin] RBAC: Authorizing `identity:get_user(user_id=1aa924e3cfe44a03bd04fd9796b5ffd0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:25.813592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e3a0e-c6cd-4916-872e-06deb1519453 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:25.815246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 590/1190] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:52:25 2022] GET /identity/v3/users/1aa924e3cfe44a03bd04fd9796b5ffd0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:25.821184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2390e36-fe27-49af-9ab6-6b92e1e825a8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:25.860138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:25.918200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2390e36-fe27-49af-9ab6-6b92e1e825a8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:25.950868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b0e88-85a5-414c-95bd-903455814538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:25.951476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b0e88-85a5-414c-95bd-903455814538 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:25.951757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b0e88-85a5-414c-95bd-903455814538 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:25.952032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b0e88-85a5-414c-95bd-903455814538 None admin] PATH_INFO: `/v3/projects/7f24a74e6d2d469cadea57d573f396bd` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:25.953165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b0e88-85a5-414c-95bd-903455814538 None admin] RBAC: Authorizing `identity:get_project(project_id=7f24a74e6d2d469cadea57d573f396bd)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:25.954591 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b0e88-85a5-414c-95bd-903455814538 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:25.956243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 601/1191] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:52:25 2022] GET /identity/v3/projects/7f24a74e6d2d469cadea57d573f396bd => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:25.962092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e3a0e-c6cd-4916-872e-06deb1519453 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:26.016410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:26.060520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e3a0e-c6cd-4916-872e-06deb1519453 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:26.094365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3bf93-d028-46e5-ba9a-a0e9769c1087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:26.095001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3bf93-d028-46e5-ba9a-a0e9769c1087 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:26.095279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3bf93-d028-46e5-ba9a-a0e9769c1087 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:26.095551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3bf93-d028-46e5-ba9a-a0e9769c1087 None admin] PATH_INFO: `/v3/projects/7f24a74e6d2d469cadea57d573f396bd/users/1aa924e3cfe44a03bd04fd9796b5ffd0/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:26.097578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3bf93-d028-46e5-ba9a-a0e9769c1087 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f24a74e6d2d469cadea57d573f396bd, user_id=1aa924e3cfe44a03bd04fd9796b5ffd0, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:26.099000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3bf93-d028-46e5-ba9a-a0e9769c1087 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:26.112196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 591/1192] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:52:25 2022] PUT /identity/v3/projects/7f24a74e6d2d469cadea57d573f396bd/users/1aa924e3cfe44a03bd04fd9796b5ffd0/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:52:27.967007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4a415-7290-499d-b01b-3e01e8557bb4 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:27.967451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4a415-7290-499d-b01b-3e01e8557bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:27.967570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4a415-7290-499d-b01b-3e01e8557bb4 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:27.968934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 602/1193] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:27.974677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8e707-427b-4a2e-8c38-d33ff136817d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:27.975114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8e707-427b-4a2e-8c38-d33ff136817d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:27.975216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8e707-427b-4a2e-8c38-d33ff136817d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:27.992891 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bed8e707-427b-4a2e-8c38-d33ff136817d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:27.994068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bed8e707-427b-4a2e-8c38-d33ff136817d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:28.062576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 592/1194] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:52:28.329896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4a415-7290-499d-b01b-3e01e8557bb4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:28.435670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4a415-7290-499d-b01b-3e01e8557bb4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:28.442589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:28.470751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90026c3d-be28-4021-8f1e-53e383851c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:28.471395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90026c3d-be28-4021-8f1e-53e383851c6b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:28.471718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90026c3d-be28-4021-8f1e-53e383851c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:28.472059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90026c3d-be28-4021-8f1e-53e383851c6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:28.482128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90026c3d-be28-4021-8f1e-53e383851c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-90026c3d-be28-4021-8f1e-53e383851c6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:52:28.486765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:52:28.487860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:52:28.488793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 603/1195] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:52:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:52:28.494102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8e707-427b-4a2e-8c38-d33ff136817d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:28.538438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:28.591862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8e707-427b-4a2e-8c38-d33ff136817d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:28.625040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d2e64-ff4c-4ea9-be96-c5d95a5fa9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:28.625633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2e64-ff4c-4ea9-be96-c5d95a5fa9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:28.625913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2e64-ff4c-4ea9-be96-c5d95a5fa9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:28.626184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2e64-ff4c-4ea9-be96-c5d95a5fa9a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:28.626842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d2e64-ff4c-4ea9-be96-c5d95a5fa9a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:52:28.627152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d2e64-ff4c-4ea9-be96-c5d95a5fa9a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:28.628552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d2e64-ff4c-4ea9-be96-c5d95a5fa9a2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:28.636359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 593/1196] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:28.642381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90026c3d-be28-4021-8f1e-53e383851c6b None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:28.745886 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90026c3d-be28-4021-8f1e-53e383851c6b None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:28.779391 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e1355-fe01-410e-9e90-5de093f603f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:28.780010 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e1355-fe01-410e-9e90-5de093f603f3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:28.780315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e1355-fe01-410e-9e90-5de093f603f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:28.780586 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e1355-fe01-410e-9e90-5de093f603f3 None admin] PATH_INFO: `/v3/users/1aa924e3cfe44a03bd04fd9796b5ffd0` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:28.781684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e1355-fe01-410e-9e90-5de093f603f3 None admin] RBAC: Authorizing `identity:get_user(user_id=1aa924e3cfe44a03bd04fd9796b5ffd0)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:28.783113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e1355-fe01-410e-9e90-5de093f603f3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:28.784739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 604/1197] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Aug 24 09:52:28 2022] GET /identity/v3/users/1aa924e3cfe44a03bd04fd9796b5ffd0 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:28.790675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d2e64-ff4c-4ea9-be96-c5d95a5fa9a2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:28.890058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d2e64-ff4c-4ea9-be96-c5d95a5fa9a2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:28.896635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:28.924013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a715af-8ee7-4170-8c36-9535b8d043ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:28.924643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a715af-8ee7-4170-8c36-9535b8d043ba None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:28.924929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a715af-8ee7-4170-8c36-9535b8d043ba None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:28.925204 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a715af-8ee7-4170-8c36-9535b8d043ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:29.008657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a715af-8ee7-4170-8c36-9535b8d043ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:29.010165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a715af-8ee7-4170-8c36-9535b8d043ba None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:29.107818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 594/1198] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:52:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:52:29.113917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e1355-fe01-410e-9e90-5de093f603f3 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:29.148484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:29.213864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e1355-fe01-410e-9e90-5de093f603f3 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:29.246944 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdb68c-cd65-4576-832f-e00dda98597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:29.247566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdb68c-cd65-4576-832f-e00dda98597f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:29.247847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdb68c-cd65-4576-832f-e00dda98597f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:29.248249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdb68c-cd65-4576-832f-e00dda98597f None admin] PATH_INFO: `/v3/projects/7f24a74e6d2d469cadea57d573f396bd` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:29.249279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebdb68c-cd65-4576-832f-e00dda98597f None admin] RBAC: Authorizing `identity:get_project(project_id=7f24a74e6d2d469cadea57d573f396bd)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:29.250723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebdb68c-cd65-4576-832f-e00dda98597f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:29.252396 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 605/1199] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Aug 24 09:52:29 2022] GET /identity/v3/projects/7f24a74e6d2d469cadea57d573f396bd => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:29.259164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a715af-8ee7-4170-8c36-9535b8d043ba None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:29.270842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:29.375576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a715af-8ee7-4170-8c36-9535b8d043ba None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:29.409007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805f2bb-f1fd-44e6-b07c-e452745f16ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:29.409665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805f2bb-f1fd-44e6-b07c-e452745f16ee None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:29.409821 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805f2bb-f1fd-44e6-b07c-e452745f16ee None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:29.410097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805f2bb-f1fd-44e6-b07c-e452745f16ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:29.410807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805f2bb-f1fd-44e6-b07c-e452745f16ee None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=7f24a74e6d2d469cadea57d573f396bd, user.id=1aa924e3cfe44a03bd04fd9796b5ffd0) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:52:29.411118 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805f2bb-f1fd-44e6-b07c-e452745f16ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:29.412588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805f2bb-f1fd-44e6-b07c-e452745f16ee None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:29.418888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 595/1200] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:52:29 2022] GET /identity/v3/role_assignments?user.id=1aa924e3cfe44a03bd04fd9796b5ffd0&scope.project.id=7f24a74e6d2d469cadea57d573f396bd&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:52:39.242730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=49 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:39.415614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=65 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:24.871681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d87dd8-4ce9-443e-915e-e5585a351f74 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:24.872292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d87dd8-4ce9-443e-915e-e5585a351f74 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:24.872292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d87dd8-4ce9-443e-915e-e5585a351f74 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:24.874051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 606/1201] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:24.881344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f47d3a-3fcb-4328-a4d4-55c449d00b9a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:24.881877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f47d3a-3fcb-4328-a4d4-55c449d00b9a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:24.882063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f47d3a-3fcb-4328-a4d4-55c449d00b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:24.907760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a6f47d3a-3fcb-4328-a4d4-55c449d00b9a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:24.909026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a6f47d3a-3fcb-4328-a4d4-55c449d00b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:24.994915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 596/1202] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:53:25.277425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d87dd8-4ce9-443e-915e-e5585a351f74 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:25.420085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d87dd8-4ce9-443e-915e-e5585a351f74 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:25.474849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238344d2-8a7d-4cf8-a09a-cf35c216ab32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:25.475495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238344d2-8a7d-4cf8-a09a-cf35c216ab32 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:25.475774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238344d2-8a7d-4cf8-a09a-cf35c216ab32 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:25.476171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238344d2-8a7d-4cf8-a09a-cf35c216ab32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:25.491104 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238344d2-8a7d-4cf8-a09a-cf35c216ab32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:25.492625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238344d2-8a7d-4cf8-a09a-cf35c216ab32 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-238344d2-8a7d-4cf8-a09a-cf35c216ab32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:25.514539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:25.515672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:25.521252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:25.521252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:25.521252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:25.521252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 607/1203] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:25.523226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f47d3a-3fcb-4328-a4d4-55c449d00b9a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:25.663410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f47d3a-3fcb-4328-a4d4-55c449d00b9a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:25.704405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:25.705405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe889b0-1b24-46ab-997a-c73a4a90a6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:25.706081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe889b0-1b24-46ab-997a-c73a4a90a6ae None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:25.706361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe889b0-1b24-46ab-997a-c73a4a90a6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:25.706650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe889b0-1b24-46ab-997a-c73a4a90a6ae None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:25.707291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe889b0-1b24-46ab-997a-c73a4a90a6ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:25.707589 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe889b0-1b24-46ab-997a-c73a4a90a6ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:25.709039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe889b0-1b24-46ab-997a-c73a4a90a6ae None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:25.723701 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 597/1204] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:25.730351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238344d2-8a7d-4cf8-a09a-cf35c216ab32 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:25.802169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:25.879188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238344d2-8a7d-4cf8-a09a-cf35c216ab32 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:25.918263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6e105-b3f7-4210-9bcd-208061d7c226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:25.919215 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e105-b3f7-4210-9bcd-208061d7c226 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:25.919686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e105-b3f7-4210-9bcd-208061d7c226 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:25.920194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e105-b3f7-4210-9bcd-208061d7c226 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:25.921093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6e105-b3f7-4210-9bcd-208061d7c226 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:25.922879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6e105-b3f7-4210-9bcd-208061d7c226 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:25.948172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 608/1205] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Aug 24 09:53:25 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:53:28.068487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe553e-2246-4132-9d7d-ba9d5ee6b168 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:28.069014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe553e-2246-4132-9d7d-ba9d5ee6b168 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:28.069014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe553e-2246-4132-9d7d-ba9d5ee6b168 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:28.071083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 598/1206] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:28.075884 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9cbe2-1eb4-4df0-8827-5be35ddeeb29 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:28.076392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9cbe2-1eb4-4df0-8827-5be35ddeeb29 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:28.076501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9cbe2-1eb4-4df0-8827-5be35ddeeb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:28.096216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-12a9cbe2-1eb4-4df0-8827-5be35ddeeb29 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:28.097422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-12a9cbe2-1eb4-4df0-8827-5be35ddeeb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:28.177766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 609/1207] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:53:28.450535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe553e-2246-4132-9d7d-ba9d5ee6b168 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:28.560438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe553e-2246-4132-9d7d-ba9d5ee6b168 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:28.599248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87b559-f719-4f11-bda2-ab078a8f9807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:28.600236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87b559-f719-4f11-bda2-ab078a8f9807 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:28.600514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87b559-f719-4f11-bda2-ab078a8f9807 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:28.600786 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87b559-f719-4f11-bda2-ab078a8f9807 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:28.614367 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87b559-f719-4f11-bda2-ab078a8f9807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0f87b559-f719-4f11-bda2-ab078a8f9807 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:53:28.619863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:28.621048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:28.621921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 599/1208] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:53:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:28.627693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9cbe2-1eb4-4df0-8827-5be35ddeeb29 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:28.677077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:28.749581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9cbe2-1eb4-4df0-8827-5be35ddeeb29 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:28.797361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb6ede-9112-409d-ab21-1a745b090405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:28.798163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb6ede-9112-409d-ab21-1a745b090405 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:28.798447 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb6ede-9112-409d-ab21-1a745b090405 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:28.798772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb6ede-9112-409d-ab21-1a745b090405 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:28.799453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb6ede-9112-409d-ab21-1a745b090405 None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:28.799754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb6ede-9112-409d-ab21-1a745b090405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:28.801260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb6ede-9112-409d-ab21-1a745b090405 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:28.815858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 610/1209] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:53:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:28.817419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87b559-f719-4f11-bda2-ab078a8f9807 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:28.886438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:28.955302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87b559-f719-4f11-bda2-ab078a8f9807 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:28.994797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64b9d4-95ec-4726-9950-5e262eeb0f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:28.995515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64b9d4-95ec-4726-9950-5e262eeb0f95 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:28.995854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64b9d4-95ec-4726-9950-5e262eeb0f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:28.996219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64b9d4-95ec-4726-9950-5e262eeb0f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:29.011447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64b9d4-95ec-4726-9950-5e262eeb0f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:29.011447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64b9d4-95ec-4726-9950-5e262eeb0f95 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ec64b9d4-95ec-4726-9950-5e262eeb0f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:29.031460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:29.032921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:29.033834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:29.033834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:29.033834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:29.033834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 600/1210] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:29.038857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb6ede-9112-409d-ab21-1a745b090405 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:29.156159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb6ede-9112-409d-ab21-1a745b090405 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:29.157765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:29.203839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c2752-6a7d-4acc-982c-b3cd955915f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:29.204599 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c2752-6a7d-4acc-982c-b3cd955915f8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:29.204879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c2752-6a7d-4acc-982c-b3cd955915f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:29.205150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c2752-6a7d-4acc-982c-b3cd955915f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:29.205863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c2752-6a7d-4acc-982c-b3cd955915f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:29.206168 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c2752-6a7d-4acc-982c-b3cd955915f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:29.207570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c2752-6a7d-4acc-982c-b3cd955915f8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:29.221080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 611/1211] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:29.229149 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64b9d4-95ec-4726-9950-5e262eeb0f95 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:29.349708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64b9d4-95ec-4726-9950-5e262eeb0f95 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:29.353591 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:29.392311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b048e-f8dc-4a69-95a5-7f43875c9409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:29.392945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b048e-f8dc-4a69-95a5-7f43875c9409 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:29.393228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b048e-f8dc-4a69-95a5-7f43875c9409 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:29.393503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b048e-f8dc-4a69-95a5-7f43875c9409 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:29.403109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b048e-f8dc-4a69-95a5-7f43875c9409 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:29.404626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b048e-f8dc-4a69-95a5-7f43875c9409 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-845b048e-f8dc-4a69-95a5-7f43875c9409 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:53:29.418756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:29.420069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:29.421376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:29.421376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:29.421376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:53:29.421376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:53:29.421376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:53:29.421376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:53:29.421376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:53:29.421376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:53:29.421376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:29.421376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:29.421376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 601/1212] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:29.426670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c2752-6a7d-4acc-982c-b3cd955915f8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:29.537867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c2752-6a7d-4acc-982c-b3cd955915f8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:29.574229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a94fe6-bc4d-4b5a-8e71-25fcd47986e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:29.574984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94fe6-bc4d-4b5a-8e71-25fcd47986e0 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:29.575389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94fe6-bc4d-4b5a-8e71-25fcd47986e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:29.575783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94fe6-bc4d-4b5a-8e71-25fcd47986e0 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:29.576588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a94fe6-bc4d-4b5a-8e71-25fcd47986e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:29.576898 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a94fe6-bc4d-4b5a-8e71-25fcd47986e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:29.578366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a94fe6-bc4d-4b5a-8e71-25fcd47986e0 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:29.611826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 612/1213] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Aug 24 09:53:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:29.612023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b048e-f8dc-4a69-95a5-7f43875c9409 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:29.729385 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b048e-f8dc-4a69-95a5-7f43875c9409 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:29.774546 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd18764-1d97-46bd-8cb3-385b2bda177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:29.775971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd18764-1d97-46bd-8cb3-385b2bda177e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:29.775971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd18764-1d97-46bd-8cb3-385b2bda177e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:29.775971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd18764-1d97-46bd-8cb3-385b2bda177e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:29.813341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:29.857248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd18764-1d97-46bd-8cb3-385b2bda177e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:29.859358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd18764-1d97-46bd-8cb3-385b2bda177e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:29.983123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 602/1214] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:53:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:53:29.989700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a94fe6-bc4d-4b5a-8e71-25fcd47986e0 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:30.013090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:30.113882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a94fe6-bc4d-4b5a-8e71-25fcd47986e0 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:30.169542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8f9c2-8b24-4707-878f-d80e8d223739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:30.170873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8f9c2-8b24-4707-878f-d80e8d223739 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:30.171876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8f9c2-8b24-4707-878f-d80e8d223739 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:30.172322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8f9c2-8b24-4707-878f-d80e8d223739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:30.183230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8f9c2-8b24-4707-878f-d80e8d223739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:30.184819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8f9c2-8b24-4707-878f-d80e8d223739 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-42e8f9c2-8b24-4707-878f-d80e8d223739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:30.200622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:30.201705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:30.202666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:30.202666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:30.202666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:30.202666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 613/1215] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:53:30.208632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd18764-1d97-46bd-8cb3-385b2bda177e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:30.290524 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:30.334615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd18764-1d97-46bd-8cb3-385b2bda177e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:30.374340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2eec6-0140-4858-9f76-47ca355fb3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:30.375004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2eec6-0140-4858-9f76-47ca355fb3da None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:30.375314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2eec6-0140-4858-9f76-47ca355fb3da None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:30.375574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2eec6-0140-4858-9f76-47ca355fb3da None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:30.376330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2eec6-0140-4858-9f76-47ca355fb3da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:30.376629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2eec6-0140-4858-9f76-47ca355fb3da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:30.378023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2eec6-0140-4858-9f76-47ca355fb3da None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:30.391066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 603/1216] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:30.397162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8f9c2-8b24-4707-878f-d80e8d223739 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:30.477174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:30.513451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8f9c2-8b24-4707-878f-d80e8d223739 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:30.551751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff5387-7ed2-4d04-8dca-8f1be77e553d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:30.552358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff5387-7ed2-4d04-8dca-8f1be77e553d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:30.552678 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff5387-7ed2-4d04-8dca-8f1be77e553d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:30.552962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff5387-7ed2-4d04-8dca-8f1be77e553d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:30.561231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff5387-7ed2-4d04-8dca-8f1be77e553d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:30.562798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff5387-7ed2-4d04-8dca-8f1be77e553d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-ebff5387-7ed2-4d04-8dca-8f1be77e553d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:53:30.574196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:30.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:30.576500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:53:30.576500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:53:30.576500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:53:30.576500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:53:30.576500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:53:30.576500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:53:30.576500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:30.576500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:30.580402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 614/1217] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:53:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:30.581739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2eec6-0140-4858-9f76-47ca355fb3da None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:30.701425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2eec6-0140-4858-9f76-47ca355fb3da None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:30.729132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:30.742505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e1fc4-6a41-490f-8b8c-1485952ebf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:30.743122 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e1fc4-6a41-490f-8b8c-1485952ebf84 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:30.743399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e1fc4-6a41-490f-8b8c-1485952ebf84 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:30.743669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e1fc4-6a41-490f-8b8c-1485952ebf84 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:30.744414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e1fc4-6a41-490f-8b8c-1485952ebf84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:30.744728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e1fc4-6a41-490f-8b8c-1485952ebf84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:30.746162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e1fc4-6a41-490f-8b8c-1485952ebf84 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:30.760056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 604/1218] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:53:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:30.766283 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff5387-7ed2-4d04-8dca-8f1be77e553d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:30.885969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff5387-7ed2-4d04-8dca-8f1be77e553d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:30.916408 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:30.923155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab843c86-14ca-4db0-8033-cdc57c2f2192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:30.923778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab843c86-14ca-4db0-8033-cdc57c2f2192 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:30.924092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab843c86-14ca-4db0-8033-cdc57c2f2192 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:30.924368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab843c86-14ca-4db0-8033-cdc57c2f2192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:30.925097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab843c86-14ca-4db0-8033-cdc57c2f2192 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=ef8d30e87c624d6180656d998afcebe7) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:30.925402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab843c86-14ca-4db0-8033-cdc57c2f2192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:30.926847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab843c86-14ca-4db0-8033-cdc57c2f2192 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:30.932301 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 615/1219] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:53:30 2022] GET /identity/v3/role_assignments?user.id=ef8d30e87c624d6180656d998afcebe7&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:53:32.929601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29cce7-0936-4d96-a508-2697807f49c3 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:32.930200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29cce7-0936-4d96-a508-2697807f49c3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:32.930293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29cce7-0936-4d96-a508-2697807f49c3 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:32.931884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 605/1220] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:53:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:32.938679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0ea59-d47d-4d63-8ed7-67f819f53eec None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:32.939146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0ea59-d47d-4d63-8ed7-67f819f53eec None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:32.939239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0ea59-d47d-4d63-8ed7-67f819f53eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:32.959248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f4f0ea59-d47d-4d63-8ed7-67f819f53eec None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:32.960161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f4f0ea59-d47d-4d63-8ed7-67f819f53eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:33.029879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 616/1221] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:53:33.296379 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29cce7-0936-4d96-a508-2697807f49c3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:33.401956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29cce7-0936-4d96-a508-2697807f49c3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:33.438227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a1f4d-4192-4ee5-a098-20b2bb890fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:33.438850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a1f4d-4192-4ee5-a098-20b2bb890fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:33.439130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a1f4d-4192-4ee5-a098-20b2bb890fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:33.439404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a1f4d-4192-4ee5-a098-20b2bb890fa0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:33.449521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a1f4d-4192-4ee5-a098-20b2bb890fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-936a1f4d-4192-4ee5-a098-20b2bb890fa0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:53:33.454418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:33.456209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:33.457418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 606/1222] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:53:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:33.461542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0ea59-d47d-4d63-8ed7-67f819f53eec None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:33.572147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0ea59-d47d-4d63-8ed7-67f819f53eec None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:33.595420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:33.607516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742055c0-c8fa-45f2-a38b-644acd4e4296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:33.608146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742055c0-c8fa-45f2-a38b-644acd4e4296 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:33.608423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742055c0-c8fa-45f2-a38b-644acd4e4296 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:33.608690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742055c0-c8fa-45f2-a38b-644acd4e4296 None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:33.609343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742055c0-c8fa-45f2-a38b-644acd4e4296 None admin] RBAC: Adding query filter params (name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:33.609643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742055c0-c8fa-45f2-a38b-644acd4e4296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:33.611006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742055c0-c8fa-45f2-a38b-644acd4e4296 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:33.618973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 617/1223] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:53:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:33.625613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a1f4d-4192-4ee5-a098-20b2bb890fa0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:33.669897 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:33.732927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a1f4d-4192-4ee5-a098-20b2bb890fa0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:33.772683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44fdb2-ce1e-47f4-ade6-2b9962f23329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:33.773389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44fdb2-ce1e-47f4-ade6-2b9962f23329 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:33.773677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44fdb2-ce1e-47f4-ade6-2b9962f23329 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:33.773952 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44fdb2-ce1e-47f4-ade6-2b9962f23329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:33.785513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44fdb2-ce1e-47f4-ade6-2b9962f23329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:33.786985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44fdb2-ce1e-47f4-ade6-2b9962f23329 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-1e44fdb2-ce1e-47f4-ade6-2b9962f23329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:33.801686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:33.802780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:33.803697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:33.803697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:33.803697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:33.803697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 607/1224] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:33.810137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742055c0-c8fa-45f2-a38b-644acd4e4296 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:33.920486 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742055c0-c8fa-45f2-a38b-644acd4e4296 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:33.959112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c4ca2-c852-4103-b445-72162b1a13ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:33.959889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c4ca2-c852-4103-b445-72162b1a13ee None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:33.960121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c4ca2-c852-4103-b445-72162b1a13ee None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:33.960359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c4ca2-c852-4103-b445-72162b1a13ee None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:33.961090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c4ca2-c852-4103-b445-72162b1a13ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:33.961412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c4ca2-c852-4103-b445-72162b1a13ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:33.962859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c4ca2-c852-4103-b445-72162b1a13ee None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:33.975700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 618/1225] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:33.981588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44fdb2-ce1e-47f4-ade6-2b9962f23329 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:34.082535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:34.091478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44fdb2-ce1e-47f4-ade6-2b9962f23329 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:34.130454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291fcc3-8f0b-4ede-a0d8-6efea9ca159d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:34.131242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291fcc3-8f0b-4ede-a0d8-6efea9ca159d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:34.131548 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291fcc3-8f0b-4ede-a0d8-6efea9ca159d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:34.131832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291fcc3-8f0b-4ede-a0d8-6efea9ca159d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:34.141118 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291fcc3-8f0b-4ede-a0d8-6efea9ca159d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:34.142665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291fcc3-8f0b-4ede-a0d8-6efea9ca159d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8291fcc3-8f0b-4ede-a0d8-6efea9ca159d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:53:34.155635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:34.157633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:34.158999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:34.158999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:34.158999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:53:34.158999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:53:34.158999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:53:34.158999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:53:34.158999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:53:34.158999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:53:34.158999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:34.158999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:34.158999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 608/1226] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:34.163017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c4ca2-c852-4103-b445-72162b1a13ee None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:34.217078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:34.274458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c4ca2-c852-4103-b445-72162b1a13ee None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:34.311005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270554a-6113-4b28-9a68-9f5c044db3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:34.311602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270554a-6113-4b28-9a68-9f5c044db3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:34.311886 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270554a-6113-4b28-9a68-9f5c044db3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:34.312187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270554a-6113-4b28-9a68-9f5c044db3c1 None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:34.312920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270554a-6113-4b28-9a68-9f5c044db3c1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:34.313236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270554a-6113-4b28-9a68-9f5c044db3c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:34.314672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270554a-6113-4b28-9a68-9f5c044db3c1 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:34.333946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 619/1227] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Aug 24 09:53:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:53:34.339781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291fcc3-8f0b-4ede-a0d8-6efea9ca159d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:34.452106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291fcc3-8f0b-4ede-a0d8-6efea9ca159d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:34.491096 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9390931-5b85-4aad-a480-2103e5dcb816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:34.491709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9390931-5b85-4aad-a480-2103e5dcb816 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:34.492123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9390931-5b85-4aad-a480-2103e5dcb816 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:34.492478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9390931-5b85-4aad-a480-2103e5dcb816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:34.506355 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9390931-5b85-4aad-a480-2103e5dcb816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:34.507831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9390931-5b85-4aad-a480-2103e5dcb816 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a9390931-5b85-4aad-a480-2103e5dcb816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:34.525044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:34.531298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:34.532488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:34.532488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:34.532488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:34.532488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 609/1228] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:34.532785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270554a-6113-4b28-9a68-9f5c044db3c1 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:34.646330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270554a-6113-4b28-9a68-9f5c044db3c1 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:34.648738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:34.686607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c919d-72f1-486d-8608-8c074c7bfddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:34.687452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c919d-72f1-486d-8608-8c074c7bfddc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:34.687829 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c919d-72f1-486d-8608-8c074c7bfddc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:34.688214 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c919d-72f1-486d-8608-8c074c7bfddc None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:34.689118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c919d-72f1-486d-8608-8c074c7bfddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:34.689510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c919d-72f1-486d-8608-8c074c7bfddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:34.691184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c919d-72f1-486d-8608-8c074c7bfddc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:34.706170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 620/1229] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:34.713167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9390931-5b85-4aad-a480-2103e5dcb816 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:34.722637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:34.823850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9390931-5b85-4aad-a480-2103e5dcb816 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:34.860004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fbd9e-1172-4943-8550-ef21cfba3d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:34.860722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fbd9e-1172-4943-8550-ef21cfba3d08 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:34.860946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fbd9e-1172-4943-8550-ef21cfba3d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:34.861209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fbd9e-1172-4943-8550-ef21cfba3d08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:34.868329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fbd9e-1172-4943-8550-ef21cfba3d08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:34.870364 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fbd9e-1172-4943-8550-ef21cfba3d08 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e66fbd9e-1172-4943-8550-ef21cfba3d08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:53:34.884157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:34.885312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:34.886418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:53:34.886418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:53:34.886418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:53:34.886418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:53:34.886418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:53:34.886418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:53:34.886418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:34.886418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:34.886418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 610/1230] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:34.891383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c919d-72f1-486d-8608-8c074c7bfddc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:35.000506 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c919d-72f1-486d-8608-8c074c7bfddc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:35.035340 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df764a5c-2d18-4266-b5ed-4898e4c62f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:35.035850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df764a5c-2d18-4266-b5ed-4898e4c62f57 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:35.036162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df764a5c-2d18-4266-b5ed-4898e4c62f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:35.036441 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df764a5c-2d18-4266-b5ed-4898e4c62f57 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:35.037134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df764a5c-2d18-4266-b5ed-4898e4c62f57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:35.037440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df764a5c-2d18-4266-b5ed-4898e4c62f57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:35.038879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df764a5c-2d18-4266-b5ed-4898e4c62f57 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:35.053451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 621/1231] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:53:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:35.060142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fbd9e-1172-4943-8550-ef21cfba3d08 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:35.133160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:35.169561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fbd9e-1172-4943-8550-ef21cfba3d08 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:35.206037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e085c62-2a8b-4f68-bab6-687e25e57534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:35.206695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e085c62-2a8b-4f68-bab6-687e25e57534 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:35.206976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e085c62-2a8b-4f68-bab6-687e25e57534 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:35.207253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e085c62-2a8b-4f68-bab6-687e25e57534 None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/ef8d30e87c624d6180656d998afcebe7/roles/080424a21d984f0eb115d19bbd53af20` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:35.226278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e085c62-2a8b-4f68-bab6-687e25e57534 None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=ef8d30e87c624d6180656d998afcebe7, role_id=080424a21d984f0eb115d19bbd53af20)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:35.227742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e085c62-2a8b-4f68-bab6-687e25e57534 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:35.241531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 611/1232] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:53:35 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/ef8d30e87c624d6180656d998afcebe7/roles/080424a21d984f0eb115d19bbd53af20 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:53:37.241856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d316820-ad36-4e1e-93d6-092d9cbf850e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:37.242526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d316820-ad36-4e1e-93d6-092d9cbf850e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:37.243282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d316820-ad36-4e1e-93d6-092d9cbf850e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:37.244309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 622/1233] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:53:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:37.251179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c6f7d-73f3-4c62-9739-3f411b57d6b3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:37.251641 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c6f7d-73f3-4c62-9739-3f411b57d6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:37.251736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c6f7d-73f3-4c62-9739-3f411b57d6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:37.271296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-310c6f7d-73f3-4c62-9739-3f411b57d6b3 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:37.272653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-310c6f7d-73f3-4c62-9739-3f411b57d6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:37.356814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 612/1234] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:53:37.642428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d316820-ad36-4e1e-93d6-092d9cbf850e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:37.672597 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:37.762063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d316820-ad36-4e1e-93d6-092d9cbf850e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:37.801948 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78a9c-2103-427d-8f4f-a7582c1b9dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:37.802650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78a9c-2103-427d-8f4f-a7582c1b9dec None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:37.802981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78a9c-2103-427d-8f4f-a7582c1b9dec None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:37.803451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78a9c-2103-427d-8f4f-a7582c1b9dec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:37.815255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf78a9c-2103-427d-8f4f-a7582c1b9dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-fdf78a9c-2103-427d-8f4f-a7582c1b9dec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:53:37.820631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:37.821805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:37.822718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 623/1235] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Aug 24 09:53:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:37.828418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c6f7d-73f3-4c62-9739-3f411b57d6b3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:37.877556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:37.938021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c6f7d-73f3-4c62-9739-3f411b57d6b3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:37.980982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca314c8-a94b-438c-9468-47b7ae2cdf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:37.981984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca314c8-a94b-438c-9468-47b7ae2cdf9e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:37.982423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca314c8-a94b-438c-9468-47b7ae2cdf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:37.983005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca314c8-a94b-438c-9468-47b7ae2cdf9e None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:37.983874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca314c8-a94b-438c-9468-47b7ae2cdf9e None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:37.984400 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca314c8-a94b-438c-9468-47b7ae2cdf9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:37.986472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca314c8-a94b-438c-9468-47b7ae2cdf9e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:37.995310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 613/1236] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Aug 24 09:53:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:38.002344 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78a9c-2103-427d-8f4f-a7582c1b9dec None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:38.109004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:38.126039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78a9c-2103-427d-8f4f-a7582c1b9dec None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:38.162226 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05e2a0-7f3a-4a05-8ce3-de5cacf1513a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:38.162860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05e2a0-7f3a-4a05-8ce3-de5cacf1513a None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:38.163141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05e2a0-7f3a-4a05-8ce3-de5cacf1513a None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:38.163408 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05e2a0-7f3a-4a05-8ce3-de5cacf1513a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:38.174885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05e2a0-7f3a-4a05-8ce3-de5cacf1513a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:38.176379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05e2a0-7f3a-4a05-8ce3-de5cacf1513a None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-3b05e2a0-7f3a-4a05-8ce3-de5cacf1513a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:38.191429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:38.192626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:38.193571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:38.193571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:38.193571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:38.193571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 624/1237] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:38.199400 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca314c8-a94b-438c-9468-47b7ae2cdf9e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:38.331221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca314c8-a94b-438c-9468-47b7ae2cdf9e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:38.341967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:38.369052 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca8d13-47bb-47db-b718-f2491e41bc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:38.370191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca8d13-47bb-47db-b718-f2491e41bc27 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:38.370476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca8d13-47bb-47db-b718-f2491e41bc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:38.370752 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca8d13-47bb-47db-b718-f2491e41bc27 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:38.371390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca8d13-47bb-47db-b718-f2491e41bc27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:38.371696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca8d13-47bb-47db-b718-f2491e41bc27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:38.373119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca8d13-47bb-47db-b718-f2491e41bc27 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:38.387511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 614/1238] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:38.393400 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05e2a0-7f3a-4a05-8ce3-de5cacf1513a None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:38.511556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05e2a0-7f3a-4a05-8ce3-de5cacf1513a None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:38.547585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705d001-4261-4790-9049-cdc79d358c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:38.548288 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705d001-4261-4790-9049-cdc79d358c36 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:38.548570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705d001-4261-4790-9049-cdc79d358c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:38.548839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705d001-4261-4790-9049-cdc79d358c36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:38.552019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:38.558474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705d001-4261-4790-9049-cdc79d358c36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:38.559326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705d001-4261-4790-9049-cdc79d358c36 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c705d001-4261-4790-9049-cdc79d358c36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:53:38.573617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:38.574788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:38.575770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:38.575770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:38.575770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:53:38.575770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:53:38.575770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:53:38.575770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:53:38.575770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:53:38.575770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:53:38.575770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:38.575770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:38.575770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 625/1239] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:38.581027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca8d13-47bb-47db-b718-f2491e41bc27 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:38.693952 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca8d13-47bb-47db-b718-f2491e41bc27 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:38.731938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3cbfb-6255-4d44-8087-dd3fc441c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:38.732573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3cbfb-6255-4d44-8087-dd3fc441c963 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:38.732879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3cbfb-6255-4d44-8087-dd3fc441c963 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:38.733151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3cbfb-6255-4d44-8087-dd3fc441c963 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:38.733965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3cbfb-6255-4d44-8087-dd3fc441c963 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:38.734270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3cbfb-6255-4d44-8087-dd3fc441c963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:38.735818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3cbfb-6255-4d44-8087-dd3fc441c963 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:38.755048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 615/1240] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Aug 24 09:53:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:53:38.761825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705d001-4261-4790-9049-cdc79d358c36 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:38.872793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705d001-4261-4790-9049-cdc79d358c36 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:38.909129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3290b-37ac-46a8-ae6a-0455b1ed27b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:38.909736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3290b-37ac-46a8-ae6a-0455b1ed27b4 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:38.910137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3290b-37ac-46a8-ae6a-0455b1ed27b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:38.910296 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3290b-37ac-46a8-ae6a-0455b1ed27b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:38.985989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:38.998493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3290b-37ac-46a8-ae6a-0455b1ed27b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:39.000780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3290b-37ac-46a8-ae6a-0455b1ed27b4 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:39.111691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 626/1241] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:53:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:53:39.119517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3cbfb-6255-4d44-8087-dd3fc441c963 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:39.150361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:39.234403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3cbfb-6255-4d44-8087-dd3fc441c963 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:39.273434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57b016-303f-4f87-aa04-cabcb51734f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:39.274069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57b016-303f-4f87-aa04-cabcb51734f4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:39.274350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57b016-303f-4f87-aa04-cabcb51734f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:39.274655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57b016-303f-4f87-aa04-cabcb51734f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:39.287280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57b016-303f-4f87-aa04-cabcb51734f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:39.289785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57b016-303f-4f87-aa04-cabcb51734f4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ea57b016-303f-4f87-aa04-cabcb51734f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:39.312328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:39.313361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:39.314263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:39.314263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:39.314263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:39.314263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 616/1242] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:53:39.320319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3290b-37ac-46a8-ae6a-0455b1ed27b4 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:39.423852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:39.432911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3290b-37ac-46a8-ae6a-0455b1ed27b4 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:39.474626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27f3d9-9e2b-4316-8599-e76999ef2407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:39.475237 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27f3d9-9e2b-4316-8599-e76999ef2407 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:39.475524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27f3d9-9e2b-4316-8599-e76999ef2407 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:39.475812 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27f3d9-9e2b-4316-8599-e76999ef2407 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:39.476621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27f3d9-9e2b-4316-8599-e76999ef2407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:39.476930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27f3d9-9e2b-4316-8599-e76999ef2407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:39.478842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27f3d9-9e2b-4316-8599-e76999ef2407 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:39.490437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 627/1243] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:39.497173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57b016-303f-4f87-aa04-cabcb51734f4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:39.580032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:39.605552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57b016-303f-4f87-aa04-cabcb51734f4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:39.643455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ec36c-9363-420b-9ecf-c8c4c08e415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:39.644121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ec36c-9363-420b-9ecf-c8c4c08e415d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:39.644448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ec36c-9363-420b-9ecf-c8c4c08e415d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:39.644812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ec36c-9363-420b-9ecf-c8c4c08e415d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:39.651967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ec36c-9363-420b-9ecf-c8c4c08e415d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:39.653601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ec36c-9363-420b-9ecf-c8c4c08e415d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-029ec36c-9363-420b-9ecf-c8c4c08e415d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:53:39.666152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:39.667417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:39.668552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:53:39.668552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:53:39.668552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:53:39.668552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:53:39.668552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:53:39.668552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:53:39.668552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:39.668552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:39.668552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 617/1244] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:53:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:53:39.673414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27f3d9-9e2b-4316-8599-e76999ef2407 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:39.783931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27f3d9-9e2b-4316-8599-e76999ef2407 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:39.823381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7dd36-ed93-4d5c-9ffd-9f3940d7ecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:39.824126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7dd36-ed93-4d5c-9ffd-9f3940d7ecd3 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:39.824364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7dd36-ed93-4d5c-9ffd-9f3940d7ecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:39.824660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7dd36-ed93-4d5c-9ffd-9f3940d7ecd3 None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:39.825455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7dd36-ed93-4d5c-9ffd-9f3940d7ecd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:39.825776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7dd36-ed93-4d5c-9ffd-9f3940d7ecd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:39.827358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7dd36-ed93-4d5c-9ffd-9f3940d7ecd3 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:39.841033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 628/1245] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:53:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:39.847277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ec36c-9363-420b-9ecf-c8c4c08e415d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:39.956789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ec36c-9363-420b-9ecf-c8c4c08e415d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:39.992549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67747c92-a9b2-4f50-8df2-cb4e28419d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:39.993156 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67747c92-a9b2-4f50-8df2-cb4e28419d32 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:39.993929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67747c92-a9b2-4f50-8df2-cb4e28419d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:39.994201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67747c92-a9b2-4f50-8df2-cb4e28419d32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:39.994929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67747c92-a9b2-4f50-8df2-cb4e28419d32 None admin] RBAC: Adding query filter params (role.id=080424a21d984f0eb115d19bbd53af20, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=ef8d30e87c624d6180656d998afcebe7) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:39.995241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67747c92-a9b2-4f50-8df2-cb4e28419d32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:39.996786 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67747c92-a9b2-4f50-8df2-cb4e28419d32 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:40.003139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 618/1246] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:53:39 2022] GET /identity/v3/role_assignments?user.id=ef8d30e87c624d6180656d998afcebe7&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=080424a21d984f0eb115d19bbd53af20 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:42.110307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018332a-ff2d-4ac6-9b57-6f6e4e103103 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:42.110872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018332a-ff2d-4ac6-9b57-6f6e4e103103 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:42.110872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018332a-ff2d-4ac6-9b57-6f6e4e103103 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:42.112198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 629/1247] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:42.118868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc04a-7a48-4b92-845a-f3ba57e712c9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:42.119317 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc04a-7a48-4b92-845a-f3ba57e712c9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:42.119447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc04a-7a48-4b92-845a-f3ba57e712c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:42.126254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:42.141656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-97dcc04a-7a48-4b92-845a-f3ba57e712c9 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:42.142893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-97dcc04a-7a48-4b92-845a-f3ba57e712c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:42.218154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 619/1248] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:53:42.503480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018332a-ff2d-4ac6-9b57-6f6e4e103103 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:42.523133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:42.622191 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018332a-ff2d-4ac6-9b57-6f6e4e103103 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:42.660009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2d39e-79d3-4746-b225-bcd94b086561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:42.660694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2d39e-79d3-4746-b225-bcd94b086561 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:42.660971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2d39e-79d3-4746-b225-bcd94b086561 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:42.661239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2d39e-79d3-4746-b225-bcd94b086561 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:42.673993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2d39e-79d3-4746-b225-bcd94b086561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c2d2d39e-79d3-4746-b225-bcd94b086561 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:53:42.678675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:42.684984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:42.686106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 630/1249] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:53:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:42.688974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcc04a-7a48-4b92-845a-f3ba57e712c9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:42.803257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcc04a-7a48-4b92-845a-f3ba57e712c9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:42.837717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:42.843202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5996b-455c-4849-a64b-1d2389de9202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:42.844833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5996b-455c-4849-a64b-1d2389de9202 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:42.844833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5996b-455c-4849-a64b-1d2389de9202 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:42.845207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5996b-455c-4849-a64b-1d2389de9202 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:42.846033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5996b-455c-4849-a64b-1d2389de9202 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:42.846538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5996b-455c-4849-a64b-1d2389de9202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:42.848138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5996b-455c-4849-a64b-1d2389de9202 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:42.857920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 620/1250] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:53:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:42.864238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2d39e-79d3-4746-b225-bcd94b086561 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:42.962201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:42.992953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2d39e-79d3-4746-b225-bcd94b086561 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:43.030201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c1616-0c0a-4538-8069-e11023c857fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:43.030753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c1616-0c0a-4538-8069-e11023c857fc None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:43.031034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c1616-0c0a-4538-8069-e11023c857fc None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:43.031382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c1616-0c0a-4538-8069-e11023c857fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:43.043567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c1616-0c0a-4538-8069-e11023c857fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:43.045318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c1616-0c0a-4538-8069-e11023c857fc None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-712c1616-0c0a-4538-8069-e11023c857fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:43.061472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:43.062662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:43.063585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:43.063585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:43.063585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:43.063585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 631/1251] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:43.068664 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5996b-455c-4849-a64b-1d2389de9202 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:43.177894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5996b-455c-4849-a64b-1d2389de9202 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:43.220476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad30091-e216-46cf-ba0c-6efcaad38125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:43.221273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad30091-e216-46cf-ba0c-6efcaad38125 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:43.221705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad30091-e216-46cf-ba0c-6efcaad38125 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:43.222097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad30091-e216-46cf-ba0c-6efcaad38125 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:43.223043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad30091-e216-46cf-ba0c-6efcaad38125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:43.223490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad30091-e216-46cf-ba0c-6efcaad38125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:43.226040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad30091-e216-46cf-ba0c-6efcaad38125 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:43.248114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 621/1252] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:53:43.251173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c1616-0c0a-4538-8069-e11023c857fc None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:43.385747 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c1616-0c0a-4538-8069-e11023c857fc None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:43.412398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:43.421184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cceb2-a950-42d7-b5e3-da631fdbb9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:43.422639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cceb2-a950-42d7-b5e3-da631fdbb9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:43.422639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cceb2-a950-42d7-b5e3-da631fdbb9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:43.422639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cceb2-a950-42d7-b5e3-da631fdbb9a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:43.431200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cceb2-a950-42d7-b5e3-da631fdbb9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:43.433257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cceb2-a950-42d7-b5e3-da631fdbb9a9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-233cceb2-a950-42d7-b5e3-da631fdbb9a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:53:43.446549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:43.452500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:43.453729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:43.453729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:43.453729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:53:43.453729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:53:43.453729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:53:43.453729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:53:43.453729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:53:43.453729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:53:43.453729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:43.453729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:43.453729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 632/1253] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:43.454419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad30091-e216-46cf-ba0c-6efcaad38125 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:43.515744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:43.568391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad30091-e216-46cf-ba0c-6efcaad38125 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:43.608517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9049f9-e97d-4410-b6e1-317961ed4601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:43.609272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9049f9-e97d-4410-b6e1-317961ed4601 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:43.609732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9049f9-e97d-4410-b6e1-317961ed4601 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:43.610057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9049f9-e97d-4410-b6e1-317961ed4601 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:43.610911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9049f9-e97d-4410-b6e1-317961ed4601 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:43.611263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9049f9-e97d-4410-b6e1-317961ed4601 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:43.612872 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9049f9-e97d-4410-b6e1-317961ed4601 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:43.637464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 622/1254] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Aug 24 09:53:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:43.643632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cceb2-a950-42d7-b5e3-da631fdbb9a9 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:43.758327 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cceb2-a950-42d7-b5e3-da631fdbb9a9 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:43.797651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03bd20-6823-434c-9e2a-f6c0078a655d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:43.798326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03bd20-6823-434c-9e2a-f6c0078a655d None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:43.798666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03bd20-6823-434c-9e2a-f6c0078a655d None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:43.799025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03bd20-6823-434c-9e2a-f6c0078a655d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:43.868568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:43.881237 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03bd20-6823-434c-9e2a-f6c0078a655d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:43.882724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03bd20-6823-434c-9e2a-f6c0078a655d None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:43.995159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 633/1255] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:53:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:53:44.001694 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9049f9-e97d-4410-b6e1-317961ed4601 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:44.114022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9049f9-e97d-4410-b6e1-317961ed4601 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:44.128130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:44.153871 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986d1f9-2bed-487a-ae7f-4769568e8c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:44.154511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986d1f9-2bed-487a-ae7f-4769568e8c4f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:44.154817 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986d1f9-2bed-487a-ae7f-4769568e8c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:44.155111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986d1f9-2bed-487a-ae7f-4769568e8c4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:44.168210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986d1f9-2bed-487a-ae7f-4769568e8c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:44.169916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986d1f9-2bed-487a-ae7f-4769568e8c4f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3986d1f9-2bed-487a-ae7f-4769568e8c4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:44.187100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:44.192184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:44.193363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:44.193363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:44.193363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:44.193363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 623/1256] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:44.194102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03bd20-6823-434c-9e2a-f6c0078a655d None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:44.298045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:44.314433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03bd20-6823-434c-9e2a-f6c0078a655d None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:44.359877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019f8f1-d420-4bea-85fd-3d7bb75414d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:44.360510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019f8f1-d420-4bea-85fd-3d7bb75414d6 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:44.360796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019f8f1-d420-4bea-85fd-3d7bb75414d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:44.361122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019f8f1-d420-4bea-85fd-3d7bb75414d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:44.362052 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019f8f1-d420-4bea-85fd-3d7bb75414d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:44.362489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019f8f1-d420-4bea-85fd-3d7bb75414d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:44.364666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019f8f1-d420-4bea-85fd-3d7bb75414d6 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:44.376536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 634/1257] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:44.382569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986d1f9-2bed-487a-ae7f-4769568e8c4f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:44.492625 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986d1f9-2bed-487a-ae7f-4769568e8c4f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:44.532563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745811f-a6f4-40d9-a309-6e461115e2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:44.533195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745811f-a6f4-40d9-a309-6e461115e2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:44.533467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745811f-a6f4-40d9-a309-6e461115e2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:44.533741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745811f-a6f4-40d9-a309-6e461115e2a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:44.541612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745811f-a6f4-40d9-a309-6e461115e2a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:44.543169 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745811f-a6f4-40d9-a309-6e461115e2a7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8745811f-a6f4-40d9-a309-6e461115e2a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:53:44.554741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:44.556107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:44.557050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:53:44.557050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:53:44.557050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:53:44.557050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:53:44.557050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:53:44.557050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:53:44.557050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:44.557050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:44.557050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 624/1258] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:53:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:44.561758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019f8f1-d420-4bea-85fd-3d7bb75414d6 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:44.678985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019f8f1-d420-4bea-85fd-3d7bb75414d6 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:44.718851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802f009-06e2-4289-8fd2-33b92ca1936b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:44.719487 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802f009-06e2-4289-8fd2-33b92ca1936b None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:44.719790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802f009-06e2-4289-8fd2-33b92ca1936b None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:44.720232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802f009-06e2-4289-8fd2-33b92ca1936b None admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:44.721032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802f009-06e2-4289-8fd2-33b92ca1936b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:44.721422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802f009-06e2-4289-8fd2-33b92ca1936b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:44.723357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802f009-06e2-4289-8fd2-33b92ca1936b None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:44.743294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 635/1259] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:53:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:44.745063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745811f-a6f4-40d9-a309-6e461115e2a7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:44.773403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:44.865265 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745811f-a6f4-40d9-a309-6e461115e2a7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:44.904786 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccdbcb-9fa5-4550-bf0a-baa9f405e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:44.905527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccdbcb-9fa5-4550-bf0a-baa9f405e8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:44.905808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccdbcb-9fa5-4550-bf0a-baa9f405e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:44.906084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccdbcb-9fa5-4550-bf0a-baa9f405e8f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:44.906838 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccdbcb-9fa5-4550-bf0a-baa9f405e8f6 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=ef8d30e87c624d6180656d998afcebe7) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:44.907149 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccdbcb-9fa5-4550-bf0a-baa9f405e8f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:44.908698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccdbcb-9fa5-4550-bf0a-baa9f405e8f6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:44.915426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 625/1260] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:53:44 2022] GET /identity/v3/role_assignments?user.id=ef8d30e87c624d6180656d998afcebe7&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:46.831236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1760c-8b73-49cf-a293-e91578e05fb7 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:46.831763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1760c-8b73-49cf-a293-e91578e05fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:46.831763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1760c-8b73-49cf-a293-e91578e05fb7 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:46.833172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 636/1261] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:46.838823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4d4ae-4361-450a-9f07-7a32af4f2c80 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:46.839262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4d4ae-4361-450a-9f07-7a32af4f2c80 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:46.839379 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4d4ae-4361-450a-9f07-7a32af4f2c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:46.857747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e4b4d4ae-4361-450a-9f07-7a32af4f2c80 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:46.858930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e4b4d4ae-4361-450a-9f07-7a32af4f2c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:46.924330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 626/1262] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:53:47.187269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1760c-8b73-49cf-a293-e91578e05fb7 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:47.198511 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:47.294941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1760c-8b73-49cf-a293-e91578e05fb7 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:47.334216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7533fb5-3dc8-4b74-913a-47c5923d7a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:47.334827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7533fb5-3dc8-4b74-913a-47c5923d7a77 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:47.335101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7533fb5-3dc8-4b74-913a-47c5923d7a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:47.335369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7533fb5-3dc8-4b74-913a-47c5923d7a77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:47.345116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7533fb5-3dc8-4b74-913a-47c5923d7a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-f7533fb5-3dc8-4b74-913a-47c5923d7a77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:53:47.349710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:47.350827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:47.351873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 637/1263] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:53:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:47.356765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4d4ae-4361-450a-9f07-7a32af4f2c80 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:47.375259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:47.453394 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4d4ae-4361-450a-9f07-7a32af4f2c80 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:47.485512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e44b33-b8be-497b-ad72-c24f2d4a4248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:47.486095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e44b33-b8be-497b-ad72-c24f2d4a4248 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:47.486370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e44b33-b8be-497b-ad72-c24f2d4a4248 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:47.486638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e44b33-b8be-497b-ad72-c24f2d4a4248 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:47.487310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e44b33-b8be-497b-ad72-c24f2d4a4248 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:47.487608 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e44b33-b8be-497b-ad72-c24f2d4a4248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:47.489254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e44b33-b8be-497b-ad72-c24f2d4a4248 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:47.497587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 627/1264] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:53:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:47.503206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7533fb5-3dc8-4b74-913a-47c5923d7a77 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:47.567673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:47.600962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7533fb5-3dc8-4b74-913a-47c5923d7a77 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:47.635320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e45e10-7b55-4a6c-9b23-29c064e034f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:47.635937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e45e10-7b55-4a6c-9b23-29c064e034f7 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:47.636249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e45e10-7b55-4a6c-9b23-29c064e034f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:47.636777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e45e10-7b55-4a6c-9b23-29c064e034f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:47.646305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e45e10-7b55-4a6c-9b23-29c064e034f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:47.647743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e45e10-7b55-4a6c-9b23-29c064e034f7 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-88e45e10-7b55-4a6c-9b23-29c064e034f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:47.662899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:47.663993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:47.665123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:47.665123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:47.665123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:47.665123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 638/1265] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:47.669989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e44b33-b8be-497b-ad72-c24f2d4a4248 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:47.750596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:47.769920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e44b33-b8be-497b-ad72-c24f2d4a4248 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:47.806155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701773a5-1259-4d9f-a2bd-7fa5893e620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:47.806760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701773a5-1259-4d9f-a2bd-7fa5893e620d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:47.807044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701773a5-1259-4d9f-a2bd-7fa5893e620d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:47.807318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701773a5-1259-4d9f-a2bd-7fa5893e620d None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:47.807955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701773a5-1259-4d9f-a2bd-7fa5893e620d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:47.808291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701773a5-1259-4d9f-a2bd-7fa5893e620d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:47.809650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701773a5-1259-4d9f-a2bd-7fa5893e620d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:47.821046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 628/1266] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:47.826855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e45e10-7b55-4a6c-9b23-29c064e034f7 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:47.923958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e45e10-7b55-4a6c-9b23-29c064e034f7 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:47.943655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:47.957443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85955e-301c-4a2e-8eff-ad6687f7a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:47.958072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85955e-301c-4a2e-8eff-ad6687f7a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:47.958367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85955e-301c-4a2e-8eff-ad6687f7a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:47.958635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85955e-301c-4a2e-8eff-ad6687f7a6e8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:47.967667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85955e-301c-4a2e-8eff-ad6687f7a6e8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:47.967667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85955e-301c-4a2e-8eff-ad6687f7a6e8 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-aa85955e-301c-4a2e-8eff-ad6687f7a6e8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:53:47.978845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:47.979969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:47.981229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:47.981229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:47.981229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:53:47.981229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:53:47.981229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:53:47.981229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:53:47.981229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:53:47.981229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:53:47.981229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:47.981229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:47.981229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 639/1267] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:53:47.987038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701773a5-1259-4d9f-a2bd-7fa5893e620d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:48.084614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701773a5-1259-4d9f-a2bd-7fa5893e620d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:48.118033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c40072-a311-4293-a60d-85517d9851fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:48.118612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c40072-a311-4293-a60d-85517d9851fa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:48.118885 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c40072-a311-4293-a60d-85517d9851fa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:48.119158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c40072-a311-4293-a60d-85517d9851fa None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:48.119869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c40072-a311-4293-a60d-85517d9851fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:48.120197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c40072-a311-4293-a60d-85517d9851fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:48.121602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c40072-a311-4293-a60d-85517d9851fa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:48.129771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:48.138398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 629/1268] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Aug 24 09:53:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:48.144027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85955e-301c-4a2e-8eff-ad6687f7a6e8 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:48.242146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85955e-301c-4a2e-8eff-ad6687f7a6e8 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:48.300531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c468480-5dca-4b9b-804a-1528f88b4f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:48.301457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c468480-5dca-4b9b-804a-1528f88b4f30 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:48.301736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c468480-5dca-4b9b-804a-1528f88b4f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:48.302078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c468480-5dca-4b9b-804a-1528f88b4f30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:48.312629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c468480-5dca-4b9b-804a-1528f88b4f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:48.314072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c468480-5dca-4b9b-804a-1528f88b4f30 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-3c468480-5dca-4b9b-804a-1528f88b4f30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:48.327442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:48.328444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:48.329442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:48.329442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:48.329442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:48.329442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 640/1269] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:48.334980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c40072-a311-4293-a60d-85517d9851fa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:48.430812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c40072-a311-4293-a60d-85517d9851fa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:48.463845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130cd5-c751-4ad3-af32-2fdbfdd25107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:48.464460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130cd5-c751-4ad3-af32-2fdbfdd25107 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:48.464737 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130cd5-c751-4ad3-af32-2fdbfdd25107 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:48.465008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130cd5-c751-4ad3-af32-2fdbfdd25107 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:48.465632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c130cd5-c751-4ad3-af32-2fdbfdd25107 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:48.465938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c130cd5-c751-4ad3-af32-2fdbfdd25107 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:48.467290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c130cd5-c751-4ad3-af32-2fdbfdd25107 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:48.478172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 630/1270] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:48.483571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c468480-5dca-4b9b-804a-1528f88b4f30 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:48.515084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:48.582235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c468480-5dca-4b9b-804a-1528f88b4f30 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:48.616365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a4520-a6af-4b40-a851-c7229259d73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:48.616913 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a4520-a6af-4b40-a851-c7229259d73e None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:48.617193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a4520-a6af-4b40-a851-c7229259d73e None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:48.617467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a4520-a6af-4b40-a851-c7229259d73e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:48.623896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a4520-a6af-4b40-a851-c7229259d73e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:48.625485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a4520-a6af-4b40-a851-c7229259d73e None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-fc6a4520-a6af-4b40-a851-c7229259d73e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:53:48.636384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:48.637453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:48.638446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:53:48.638446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:53:48.638446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:53:48.638446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:53:48.638446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:53:48.638446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:53:48.638446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:48.638446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:48.638446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 641/1271] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:53:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:48.643235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130cd5-c751-4ad3-af32-2fdbfdd25107 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:48.700954 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:48.743232 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130cd5-c751-4ad3-af32-2fdbfdd25107 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:48.776675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bde6b-36d9-49f9-8b38-051cbe94a1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:48.777276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bde6b-36d9-49f9-8b38-051cbe94a1ca None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:48.777557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bde6b-36d9-49f9-8b38-051cbe94a1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:48.777830 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bde6b-36d9-49f9-8b38-051cbe94a1ca None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:48.778521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bde6b-36d9-49f9-8b38-051cbe94a1ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:48.778822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bde6b-36d9-49f9-8b38-051cbe94a1ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:48.780267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bde6b-36d9-49f9-8b38-051cbe94a1ca None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:48.794184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 631/1272] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:53:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:48.800286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a4520-a6af-4b40-a851-c7229259d73e None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:48.901461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:48.910775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a4520-a6af-4b40-a851-c7229259d73e None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:48.945160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db57b3-4df9-49bd-a7a9-c4236587b11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:48.945811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db57b3-4df9-49bd-a7a9-c4236587b11f None admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:48.946069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db57b3-4df9-49bd-a7a9-c4236587b11f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:48.946337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db57b3-4df9-49bd-a7a9-c4236587b11f None admin] PATH_INFO: `/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/ef8d30e87c624d6180656d998afcebe7/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:48.948274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db57b3-4df9-49bd-a7a9-c4236587b11f None admin] RBAC: Authorizing `identity:create_grant(project_id=3955b2c7daac4b47a5a38418ec79a089, user_id=ef8d30e87c624d6180656d998afcebe7, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:48.949759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db57b3-4df9-49bd-a7a9-c4236587b11f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:48.963481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 642/1273] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Aug 24 09:53:48 2022] PUT /identity/v3/projects/3955b2c7daac4b47a5a38418ec79a089/users/ef8d30e87c624d6180656d998afcebe7/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:53:50.845976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bf9ab-9be9-4de5-bd80-9aed1f5adafa None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:50.846529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bf9ab-9be9-4de5-bd80-9aed1f5adafa None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:50.846529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bf9ab-9be9-4de5-bd80-9aed1f5adafa None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:50.847855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 632/1274] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:50.853066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbba7e9-9c54-4dd9-ae49-7352e03caa93 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:50.853470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbba7e9-9c54-4dd9-ae49-7352e03caa93 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:50.853612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbba7e9-9c54-4dd9-ae49-7352e03caa93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:50.871976 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1dbba7e9-9c54-4dd9-ae49-7352e03caa93 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:50.873168 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1dbba7e9-9c54-4dd9-ae49-7352e03caa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:50.936610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 643/1275] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:53:51.203601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bf9ab-9be9-4de5-bd80-9aed1f5adafa None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:51.313190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bf9ab-9be9-4de5-bd80-9aed1f5adafa None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:51.330805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:51.348226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc252427-c571-4b4c-a8e7-a6f0f9d91f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:51.349306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc252427-c571-4b4c-a8e7-a6f0f9d91f1b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:51.349775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc252427-c571-4b4c-a8e7-a6f0f9d91f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:51.350249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc252427-c571-4b4c-a8e7-a6f0f9d91f1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:51.364541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc252427-c571-4b4c-a8e7-a6f0f9d91f1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-cc252427-c571-4b4c-a8e7-a6f0f9d91f1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:53:51.369223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:51.370327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:51.371717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 633/1276] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Aug 24 09:53:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:51.376341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbba7e9-9c54-4dd9-ae49-7352e03caa93 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:51.454323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:51.473624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbba7e9-9c54-4dd9-ae49-7352e03caa93 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:51.507331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fa1a8-4e4c-4673-8aaa-d0354d9bbfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:51.507920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fa1a8-4e4c-4673-8aaa-d0354d9bbfdb None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:51.508230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fa1a8-4e4c-4673-8aaa-d0354d9bbfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:51.508506 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fa1a8-4e4c-4673-8aaa-d0354d9bbfdb None admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:51.509167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fa1a8-4e4c-4673-8aaa-d0354d9bbfdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:51.509475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fa1a8-4e4c-4673-8aaa-d0354d9bbfdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:51.510842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fa1a8-4e4c-4673-8aaa-d0354d9bbfdb None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:51.518652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 644/1277] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Aug 24 09:53:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:51.524360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc252427-c571-4b4c-a8e7-a6f0f9d91f1b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:51.623278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc252427-c571-4b4c-a8e7-a6f0f9d91f1b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:51.656392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afb1ae-7a20-495f-9150-244ad8cd343d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:51.657009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afb1ae-7a20-495f-9150-244ad8cd343d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:51.657290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afb1ae-7a20-495f-9150-244ad8cd343d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:51.657565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afb1ae-7a20-495f-9150-244ad8cd343d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:51.668249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16afb1ae-7a20-495f-9150-244ad8cd343d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:51.669685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16afb1ae-7a20-495f-9150-244ad8cd343d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-16afb1ae-7a20-495f-9150-244ad8cd343d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:51.685113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:51.686281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:51.687431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:51.687431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:51.687431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:51.687431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 634/1278] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:51.692587 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fa1a8-4e4c-4673-8aaa-d0354d9bbfdb None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:51.791999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fa1a8-4e4c-4673-8aaa-d0354d9bbfdb None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:51.825400 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170780da-ddc2-432e-8e2b-384b672dbc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:51.826030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170780da-ddc2-432e-8e2b-384b672dbc56 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:51.826310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170780da-ddc2-432e-8e2b-384b672dbc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:51.826581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170780da-ddc2-432e-8e2b-384b672dbc56 None admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:51.827249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170780da-ddc2-432e-8e2b-384b672dbc56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:51.827552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170780da-ddc2-432e-8e2b-384b672dbc56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:51.828970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170780da-ddc2-432e-8e2b-384b672dbc56 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:51.840693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 645/1279] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:53:51.846305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afb1ae-7a20-495f-9150-244ad8cd343d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:51.876906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:51.945506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afb1ae-7a20-495f-9150-244ad8cd343d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:51.977446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d342ad5-e4dc-4301-9126-5e98bfe0cc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:51.978058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d342ad5-e4dc-4301-9126-5e98bfe0cc81 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:51.978345 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d342ad5-e4dc-4301-9126-5e98bfe0cc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:51.978606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d342ad5-e4dc-4301-9126-5e98bfe0cc81 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:51.985824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d342ad5-e4dc-4301-9126-5e98bfe0cc81 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:51.987518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d342ad5-e4dc-4301-9126-5e98bfe0cc81 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-9d342ad5-e4dc-4301-9126-5e98bfe0cc81 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:53:51.998971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:52.000116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:52.001193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:52.001193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:52.001193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:53:52.001193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:53:52.001193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:53:52.001193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:53:52.001193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:53:52.001193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:53:52.001193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:53:52.001193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:52.001193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 635/1280] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:52.006548 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170780da-ddc2-432e-8e2b-384b672dbc56 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:52.010440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:52.103684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170780da-ddc2-432e-8e2b-384b672dbc56 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:52.138115 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b268c-3802-4162-af21-3449452709dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:52.138703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b268c-3802-4162-af21-3449452709dd None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:52.138980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b268c-3802-4162-af21-3449452709dd None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:52.139248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b268c-3802-4162-af21-3449452709dd None admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:52.139969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b268c-3802-4162-af21-3449452709dd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:52.140303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b268c-3802-4162-af21-3449452709dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:52.141727 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b268c-3802-4162-af21-3449452709dd None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:52.158274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 646/1281] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Aug 24 09:53:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:52.164467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d342ad5-e4dc-4301-9126-5e98bfe0cc81 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:52.253794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:52.265241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d342ad5-e4dc-4301-9126-5e98bfe0cc81 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:52.298270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55107bdf-f9dc-41f6-90b6-ee695d158951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:52.298868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55107bdf-f9dc-41f6-90b6-ee695d158951 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:52.299147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55107bdf-f9dc-41f6-90b6-ee695d158951 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:52.299416 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55107bdf-f9dc-41f6-90b6-ee695d158951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:52.365224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55107bdf-f9dc-41f6-90b6-ee695d158951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:52.366592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55107bdf-f9dc-41f6-90b6-ee695d158951 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:52.450625 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:52.461253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 636/1282] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Aug 24 09:53:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:53:52.466906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b268c-3802-4162-af21-3449452709dd None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:52.536512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:52.576805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b268c-3802-4162-af21-3449452709dd None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:52.611628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec6e34-3ca9-47ba-9323-455e3d716524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:52.612284 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec6e34-3ca9-47ba-9323-455e3d716524 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:52.612563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec6e34-3ca9-47ba-9323-455e3d716524 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:52.612831 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec6e34-3ca9-47ba-9323-455e3d716524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:52.623477 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec6e34-3ca9-47ba-9323-455e3d716524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:52.624924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec6e34-3ca9-47ba-9323-455e3d716524 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-c4ec6e34-3ca9-47ba-9323-455e3d716524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:53:52.640633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:52.641840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:53:52.642979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:53:52.642979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:53:52.642979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:52.642979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 647/1283] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:52.648066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55107bdf-f9dc-41f6-90b6-ee695d158951 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:52.747549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55107bdf-f9dc-41f6-90b6-ee695d158951 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:52.780525 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0208ba-d938-4507-859b-2a7d51e8de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:52.781125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0208ba-d938-4507-859b-2a7d51e8de0e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:52.781400 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0208ba-d938-4507-859b-2a7d51e8de0e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:52.781670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0208ba-d938-4507-859b-2a7d51e8de0e None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:52.782311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0208ba-d938-4507-859b-2a7d51e8de0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:52.782614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0208ba-d938-4507-859b-2a7d51e8de0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:52.783987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0208ba-d938-4507-859b-2a7d51e8de0e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:52.797118 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 637/1284] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Aug 24 09:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:52.802928 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec6e34-3ca9-47ba-9323-455e3d716524 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:52.902524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec6e34-3ca9-47ba-9323-455e3d716524 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:52.915540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:52.935681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dccf2-57ed-47a9-99f2-18c0364fb565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:52.936319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dccf2-57ed-47a9-99f2-18c0364fb565 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:52.936597 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dccf2-57ed-47a9-99f2-18c0364fb565 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:52.936866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dccf2-57ed-47a9-99f2-18c0364fb565 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:52.943889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706dccf2-57ed-47a9-99f2-18c0364fb565 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:52.945346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706dccf2-57ed-47a9-99f2-18c0364fb565 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-706dccf2-57ed-47a9-99f2-18c0364fb565 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:53:52.956471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:52.957606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:52.958916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:53:52.958916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:53:52.958916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:53:52.958916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:53:52.958916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:53:52.958916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:53:52.958916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:53:52.958916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 09:53:52.958916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 648/1285] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Aug 24 09:53:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:52.963353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0208ba-d938-4507-859b-2a7d51e8de0e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:53.013474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:53.063956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0208ba-d938-4507-859b-2a7d51e8de0e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:53.096562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d028d5-cb2b-4874-badd-7a1a974b2871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:53.097684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d028d5-cb2b-4874-badd-7a1a974b2871 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:53.097963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d028d5-cb2b-4874-badd-7a1a974b2871 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:53.098235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d028d5-cb2b-4874-badd-7a1a974b2871 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:53.098921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d028d5-cb2b-4874-badd-7a1a974b2871 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:53.099229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d028d5-cb2b-4874-badd-7a1a974b2871 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:53.100689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d028d5-cb2b-4874-badd-7a1a974b2871 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:53.113516 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 638/1286] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Aug 24 09:53:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:53.119753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dccf2-57ed-47a9-99f2-18c0364fb565 None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:53.216660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dccf2-57ed-47a9-99f2-18c0364fb565 None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:53.250428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b0251-8890-49f3-81ac-5a2dd0fe7f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:53.251072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b0251-8890-49f3-81ac-5a2dd0fe7f59 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:53.251387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b0251-8890-49f3-81ac-5a2dd0fe7f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:53.251700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b0251-8890-49f3-81ac-5a2dd0fe7f59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:53.252527 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b0251-8890-49f3-81ac-5a2dd0fe7f59 None admin] RBAC: Adding query filter params (role.id=3db28152165c45b3bc5ac611f2a8a612, scope.project.id=3955b2c7daac4b47a5a38418ec79a089, user.id=ef8d30e87c624d6180656d998afcebe7) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:53.252865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b0251-8890-49f3-81ac-5a2dd0fe7f59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:53.254998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b0251-8890-49f3-81ac-5a2dd0fe7f59 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:53.261535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 649/1287] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Aug 24 09:53:53 2022] GET /identity/v3/role_assignments?user.id=ef8d30e87c624d6180656d998afcebe7&scope.project.id=3955b2c7daac4b47a5a38418ec79a089&role.id=3db28152165c45b3bc5ac611f2a8a612 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:55.147929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721e66e-3c09-43b9-a981-b831c980fc90 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:55.148488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721e66e-3c09-43b9-a981-b831c980fc90 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:55.148488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721e66e-3c09-43b9-a981-b831c980fc90 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:55.149785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 639/1288] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:55.154867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7767a87-c9b4-45aa-8295-680ed0c0a2b7 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:55.155338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7767a87-c9b4-45aa-8295-680ed0c0a2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:55.155452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7767a87-c9b4-45aa-8295-680ed0c0a2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:55.173608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f7767a87-c9b4-45aa-8295-680ed0c0a2b7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:55.175426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f7767a87-c9b4-45aa-8295-680ed0c0a2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:55.203922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:55.241427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 650/1289] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:53:55.513846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721e66e-3c09-43b9-a981-b831c980fc90 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:55.615336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721e66e-3c09-43b9-a981-b831c980fc90 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:55.625922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:55.649723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993247e-9c11-4792-882b-18c0594f8106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:55.650339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993247e-9c11-4792-882b-18c0594f8106 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:55.650621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993247e-9c11-4792-882b-18c0594f8106 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:55.650890 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993247e-9c11-4792-882b-18c0594f8106 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:55.651467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993247e-9c11-4792-882b-18c0594f8106 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:55.652850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993247e-9c11-4792-882b-18c0594f8106 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c993247e-9c11-4792-882b-18c0594f8106 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:53:55.662128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:53:55.663341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:53:55.664285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 640/1290] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Aug 24 09:53:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:53:55.669121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7767a87-c9b4-45aa-8295-680ed0c0a2b7 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:55.765297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7767a87-c9b4-45aa-8295-680ed0c0a2b7 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:55.799200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfa43d-437c-48cb-a405-78158bdcdeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:55.799793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfa43d-437c-48cb-a405-78158bdcdeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:55.800096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfa43d-437c-48cb-a405-78158bdcdeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:55.800370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfa43d-437c-48cb-a405-78158bdcdeb9 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:55.800995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfa43d-437c-48cb-a405-78158bdcdeb9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:55.801295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfa43d-437c-48cb-a405-78158bdcdeb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:55.802667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfa43d-437c-48cb-a405-78158bdcdeb9 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:55.807393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 651/1291] 10.222.0.86 () {56 vars in 1116 bytes} [Wed Aug 24 09:53:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:55.813104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993247e-9c11-4792-882b-18c0594f8106 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:55.915642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993247e-9c11-4792-882b-18c0594f8106 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:55.951648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc711dc5-a66e-41db-8bf6-b026624ba036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:55.952271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc711dc5-a66e-41db-8bf6-b026624ba036 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:55.952557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc711dc5-a66e-41db-8bf6-b026624ba036 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:55.952833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc711dc5-a66e-41db-8bf6-b026624ba036 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:55.953459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc711dc5-a66e-41db-8bf6-b026624ba036 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:55.953765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc711dc5-a66e-41db-8bf6-b026624ba036 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:55.955774 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc711dc5-a66e-41db-8bf6-b026624ba036 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:55.960365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 641/1292] 10.222.0.86 () {56 vars in 1116 bytes} [Wed Aug 24 09:53:55 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:53:57.835822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3425bc7-53fd-4bb4-b05c-c3a4de7a3cdb None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:57.836365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3425bc7-53fd-4bb4-b05c-c3a4de7a3cdb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:57.836483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3425bc7-53fd-4bb4-b05c-c3a4de7a3cdb None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:57.837761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 652/1293] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:57.843031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea97b71-0e13-4270-bd4d-897f9aee577e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:57.843447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea97b71-0e13-4270-bd4d-897f9aee577e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:57.843571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea97b71-0e13-4270-bd4d-897f9aee577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:57.861995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9ea97b71-0e13-4270-bd4d-897f9aee577e None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:57.863167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9ea97b71-0e13-4270-bd4d-897f9aee577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:57.892298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:57.929084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 642/1294] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:53:58.194614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3425bc7-53fd-4bb4-b05c-c3a4de7a3cdb None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:58.225542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:58.306900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3425bc7-53fd-4bb4-b05c-c3a4de7a3cdb None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:58.343665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6c814-96fa-4dd8-a7e0-4e2932e64588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:58.344295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c814-96fa-4dd8-a7e0-4e2932e64588 None admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:58.344578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c814-96fa-4dd8-a7e0-4e2932e64588 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:58.345361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c814-96fa-4dd8-a7e0-4e2932e64588 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:58.346012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6c814-96fa-4dd8-a7e0-4e2932e64588 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:58.347398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6c814-96fa-4dd8-a7e0-4e2932e64588 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:58.363185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 653/1295] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Aug 24 09:53:58 2022] POST /identity/v3/services => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:54:00.270690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7177c8-31e9-4768-b9e8-d17ce78ec56d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:00.271243 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7177c8-31e9-4768-b9e8-d17ce78ec56d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:00.271243 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7177c8-31e9-4768-b9e8-d17ce78ec56d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:00.272667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 643/1296] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:00.277856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf67e7e-beb6-4372-8e6f-f94c0e78d4e8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:00.278257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf67e7e-beb6-4372-8e6f-f94c0e78d4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:00.278394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf67e7e-beb6-4372-8e6f-f94c0e78d4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:00.296760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fdf67e7e-beb6-4372-8e6f-f94c0e78d4e8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:00.298449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fdf67e7e-beb6-4372-8e6f-f94c0e78d4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:00.369105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 654/1297] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:54:00 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:54:00.638526 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7177c8-31e9-4768-b9e8-d17ce78ec56d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:00.738826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7177c8-31e9-4768-b9e8-d17ce78ec56d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:00.772495 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8780f-f191-40c7-8266-b0d1d6b607c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:00.773117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8780f-f191-40c7-8266-b0d1d6b607c1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:00.773395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8780f-f191-40c7-8266-b0d1d6b607c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:00.773754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8780f-f191-40c7-8266-b0d1d6b607c1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:00.774354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8780f-f191-40c7-8266-b0d1d6b607c1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:00.775749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8780f-f191-40c7-8266-b0d1d6b607c1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7cd8780f-f191-40c7-8266-b0d1d6b607c1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:54:00.785064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:54:00.786049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:54:00.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 644/1298] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Aug 24 09:54:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:54:00.792169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf67e7e-beb6-4372-8e6f-f94c0e78d4e8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:00.810095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:00.889474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf67e7e-beb6-4372-8e6f-f94c0e78d4e8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:00.922569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dda19-03ca-4d17-827a-5e78cd672a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:00.923161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dda19-03ca-4d17-827a-5e78cd672a3f None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:00.923438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dda19-03ca-4d17-827a-5e78cd672a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:00.923708 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dda19-03ca-4d17-827a-5e78cd672a3f None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:00.924358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14dda19-03ca-4d17-827a-5e78cd672a3f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:54:00.924659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14dda19-03ca-4d17-827a-5e78cd672a3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:00.926027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14dda19-03ca-4d17-827a-5e78cd672a3f None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:00.930992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 655/1299] 10.222.0.86 () {56 vars in 1116 bytes} [Wed Aug 24 09:54:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:54:00.937027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8780f-f191-40c7-8266-b0d1d6b607c1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:00.968400 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:01.036536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8780f-f191-40c7-8266-b0d1d6b607c1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:01.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1caaf-d281-47f5-9b12-e2d4c5e288d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:01.070634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1caaf-d281-47f5-9b12-e2d4c5e288d3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:01.070907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1caaf-d281-47f5-9b12-e2d4c5e288d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:01.071179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1caaf-d281-47f5-9b12-e2d4c5e288d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:01.071929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1caaf-d281-47f5-9b12-e2d4c5e288d3 None admin] RBAC: Adding query filter params (interface=public, service_id=919656f545d6453e8f475bb4cb84007f, region_id=RegionOne) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:54:01.072311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1caaf-d281-47f5-9b12-e2d4c5e288d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:01.073707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1caaf-d281-47f5-9b12-e2d4c5e288d3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:01.079136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 645/1300] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Aug 24 09:54:00 2022] GET /identity/v3/endpoints?service_id=919656f545d6453e8f475bb4cb84007f&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:54:01.084822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dda19-03ca-4d17-827a-5e78cd672a3f None admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:01.164564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:01.182412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dda19-03ca-4d17-827a-5e78cd672a3f None admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:01.215978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94b5c7-f5bc-43ce-ae6f-a7dd0e436a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:01.216596 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94b5c7-f5bc-43ce-ae6f-a7dd0e436a87 None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:01.216877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94b5c7-f5bc-43ce-ae6f-a7dd0e436a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:01.217152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94b5c7-f5bc-43ce-ae6f-a7dd0e436a87 None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:01.217756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94b5c7-f5bc-43ce-ae6f-a7dd0e436a87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:01.219150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94b5c7-f5bc-43ce-ae6f-a7dd0e436a87 None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:01.225777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 656/1301] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Aug 24 09:54:01 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:54:03.112778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf0a68-32ea-4623-8d3b-b34f436e36aa None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:03.113277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf0a68-32ea-4623-8d3b-b34f436e36aa None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:03.113277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf0a68-32ea-4623-8d3b-b34f436e36aa None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:03.114603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 646/1302] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:03.119878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0260a-613b-4b77-bd7a-259517110326 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:03.120321 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0260a-613b-4b77-bd7a-259517110326 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:03.120461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0260a-613b-4b77-bd7a-259517110326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:03.138648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fae0260a-613b-4b77-bd7a-259517110326 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:03.139852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fae0260a-613b-4b77-bd7a-259517110326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:03.204965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 657/1303] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:54:03 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:54:03.489516 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf0a68-32ea-4623-8d3b-b34f436e36aa None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:03.581918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:03.588685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf0a68-32ea-4623-8d3b-b34f436e36aa None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:03.621799 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff045-f5db-4aad-a01c-14e17b490aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:03.622409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff045-f5db-4aad-a01c-14e17b490aeb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:03.622684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff045-f5db-4aad-a01c-14e17b490aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:03.622950 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff045-f5db-4aad-a01c-14e17b490aeb None admin] PATH_INFO: `/v3/services/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:03.623518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aff045-f5db-4aad-a01c-14e17b490aeb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:03.624938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aff045-f5db-4aad-a01c-14e17b490aeb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-11aff045-f5db-4aad-a01c-14e17b490aeb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:54:03.633272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:54:03.634444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 24 09:54:03.635680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 647/1304] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Aug 24 09:54:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:54:03.640371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0260a-613b-4b77-bd7a-259517110326 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:03.714887 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:03.738677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0260a-613b-4b77-bd7a-259517110326 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:03.772194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8bf74d-dcf2-4e79-a956-6eccb820661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:03.772798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bf74d-dcf2-4e79-a956-6eccb820661c None admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:03.773074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bf74d-dcf2-4e79-a956-6eccb820661c None admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:03.773346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bf74d-dcf2-4e79-a956-6eccb820661c None admin] PATH_INFO: `/v3/services` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:03.773970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8bf74d-dcf2-4e79-a956-6eccb820661c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:54:03.774272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8bf74d-dcf2-4e79-a956-6eccb820661c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:03.775643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8bf74d-dcf2-4e79-a956-6eccb820661c None admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:03.781401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 658/1305] 10.222.0.86 () {56 vars in 1116 bytes} [Wed Aug 24 09:54:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:54:03.792909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff045-f5db-4aad-a01c-14e17b490aeb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:03.896166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff045-f5db-4aad-a01c-14e17b490aeb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:03.929523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07317ba-5315-4677-b78b-029c7248ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:03.930123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07317ba-5315-4677-b78b-029c7248ac4a None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:03.930407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07317ba-5315-4677-b78b-029c7248ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:03.930684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07317ba-5315-4677-b78b-029c7248ac4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:03.931335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07317ba-5315-4677-b78b-029c7248ac4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:03.932735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07317ba-5315-4677-b78b-029c7248ac4a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:03.951702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 648/1306] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 09:54:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:54:13.777999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=30 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:13.947882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:07.962235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae876-681d-4b3c-92ba-24cb924a1bef None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:07.963573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae876-681d-4b3c-92ba-24cb924a1bef None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:07.964213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae876-681d-4b3c-92ba-24cb924a1bef None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:07.974599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379edd71-91ff-46ff-978c-c28a65fe68b7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:07.975659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379edd71-91ff-46ff-978c-c28a65fe68b7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:07.976352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379edd71-91ff-46ff-978c-c28a65fe68b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:07.983650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 659/1307] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:07 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:08.180752 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-379edd71-91ff-46ff-978c-c28a65fe68b7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:08.182481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-379edd71-91ff-46ff-978c-c28a65fe68b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:08.313351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 649/1308] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:08.606706 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454db57-905a-4862-b20f-bac3e44fd944 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:08.607172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454db57-905a-4862-b20f-bac3e44fd944 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:08.607667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454db57-905a-4862-b20f-bac3e44fd944 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:08.609567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 660/1309] 10.222.0.86 () {54 vars in 965 bytes} [Wed Aug 24 09:55:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:08.615176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b03d72-7151-469a-b1f1-f8e8d0b6e7ed None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:08.615585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b03d72-7151-469a-b1f1-f8e8d0b6e7ed None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:08.615971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b03d72-7151-469a-b1f1-f8e8d0b6e7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:08.698340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b2b03d72-7151-469a-b1f1-f8e8d0b6e7ed None None] MFA Rules not processed for user `27cc091f3b094cb1a1e2650a3fbd7671`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:08.699726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b2b03d72-7151-469a-b1f1-f8e8d0b6e7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:08.721822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:08.805399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 650/1310] 10.222.0.86 () {58 vars in 1062 bytes} [Wed Aug 24 09:55:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:08.814782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454db57-905a-4862-b20f-bac3e44fd944 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:08.816576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8454db57-905a-4862-b20f-bac3e44fd944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:08.939465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454db57-905a-4862-b20f-bac3e44fd944 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:09.020370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf581b-c4d2-4a87-a540-c34b38d1c972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:09.021045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf581b-c4d2-4a87-a540-c34b38d1c972 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:09.021332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf581b-c4d2-4a87-a540-c34b38d1c972 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:09.021524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf581b-c4d2-4a87-a540-c34b38d1c972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:09.023130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f9cf581b-c4d2-4a87-a540-c34b38d1c972 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:09.087025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:09.161826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf581b-c4d2-4a87-a540-c34b38d1c972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:09.165460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf581b-c4d2-4a87-a540-c34b38d1c972 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:09.205232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 661/1311] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:11.523867 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e901d-16eb-4d18-8fff-995d230644f3 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:11.524405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e901d-16eb-4d18-8fff-995d230644f3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:11.524900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e901d-16eb-4d18-8fff-995d230644f3 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:11.527904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 651/1312] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:11.533438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991c26e-9f67-4bab-87e1-26be2a4350ed None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:11.533881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991c26e-9f67-4bab-87e1-26be2a4350ed None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:11.534042 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991c26e-9f67-4bab-87e1-26be2a4350ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:11.606506 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1991c26e-9f67-4bab-87e1-26be2a4350ed None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:11.607713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1991c26e-9f67-4bab-87e1-26be2a4350ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:11.703372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 662/1313] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:11.998215 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7269b4c-4b6f-454d-8a8d-98f0acaa049d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:11.998492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7269b4c-4b6f-454d-8a8d-98f0acaa049d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:11.998758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7269b4c-4b6f-454d-8a8d-98f0acaa049d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:12.000130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 652/1314] 10.222.0.86 () {54 vars in 965 bytes} [Wed Aug 24 09:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:12.005981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d58ef-2786-4eef-9a87-24c9a4ad4596 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:12.006269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d58ef-2786-4eef-9a87-24c9a4ad4596 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:12.006582 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d58ef-2786-4eef-9a87-24c9a4ad4596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:12.018303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:12.084000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ed6d58ef-2786-4eef-9a87-24c9a4ad4596 None None] MFA Rules not processed for user `27cc091f3b094cb1a1e2650a3fbd7671`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:12.085374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ed6d58ef-2786-4eef-9a87-24c9a4ad4596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:12.160402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 663/1315] 10.222.0.86 () {58 vars in 1062 bytes} [Wed Aug 24 09:55:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:12.169760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7269b4c-4b6f-454d-8a8d-98f0acaa049d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:12.171291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b7269b4c-4b6f-454d-8a8d-98f0acaa049d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:12.273837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7269b4c-4b6f-454d-8a8d-98f0acaa049d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:12.344332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:12.346915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649e1df-b248-47e2-9f50-877119d78be0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:12.347539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649e1df-b248-47e2-9f50-877119d78be0 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:12.348038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649e1df-b248-47e2-9f50-877119d78be0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:12.348572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649e1df-b248-47e2-9f50-877119d78be0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:12.352216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1649e1df-b248-47e2-9f50-877119d78be0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:12.460009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649e1df-b248-47e2-9f50-877119d78be0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:12.461471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649e1df-b248-47e2-9f50-877119d78be0 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:12.481083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 653/1316] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:14.925518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f30610-e056-4e62-af51-669df4987f4b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:14.926163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f30610-e056-4e62-af51-669df4987f4b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:14.926163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f30610-e056-4e62-af51-669df4987f4b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:14.927426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 664/1317] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:14.933366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63951b-0898-42e9-971d-0303f9f20f74 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:14.933801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63951b-0898-42e9-971d-0303f9f20f74 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:14.933930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63951b-0898-42e9-971d-0303f9f20f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:15.000285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1f63951b-0898-42e9-971d-0303f9f20f74 None None] MFA Rules not processed for user `7d7bdddb7df24e8a9615b6909130f4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:15.001479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1f63951b-0898-42e9-971d-0303f9f20f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:15.053895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:15.098351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 654/1318] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:15.382494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f30610-e056-4e62-af51-669df4987f4b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:15.384026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f5f30610-e056-4e62-af51-669df4987f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:15.486199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f30610-e056-4e62-af51-669df4987f4b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:15.489073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:15.559654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980bcc3-4b45-4d69-8c38-b54dbd78116d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:15.560287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980bcc3-4b45-4d69-8c38-b54dbd78116d service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:15.560569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980bcc3-4b45-4d69-8c38-b54dbd78116d service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:15.560842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980bcc3-4b45-4d69-8c38-b54dbd78116d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:15.562433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c980bcc3-4b45-4d69-8c38-b54dbd78116d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:15.669657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980bcc3-4b45-4d69-8c38-b54dbd78116d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:15.671018 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980bcc3-4b45-4d69-8c38-b54dbd78116d service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:15.688615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 665/1319] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:15 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:18.079220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3189-8657-429d-a9ed-745966a3502b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:18.079748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3189-8657-429d-a9ed-745966a3502b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:18.079748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3189-8657-429d-a9ed-745966a3502b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:18.081073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 655/1320] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:18.086293 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3305ca4-67ac-4c01-bab6-e9ad3a75ea48 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:18.086726 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3305ca4-67ac-4c01-bab6-e9ad3a75ea48 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:18.086867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3305ca4-67ac-4c01-bab6-e9ad3a75ea48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:18.127460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:18.152033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a3305ca4-67ac-4c01-bab6-e9ad3a75ea48 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:18.153254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a3305ca4-67ac-4c01-bab6-e9ad3a75ea48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:18.247712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 666/1321] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:18.529727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e3189-8657-429d-a9ed-745966a3502b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:18.531258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-364e3189-8657-429d-a9ed-745966a3502b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:18.634245 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e3189-8657-429d-a9ed-745966a3502b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:18.705000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262d2ec-f7fe-4a3c-8136-c56a7e28cbca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:18.705623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262d2ec-f7fe-4a3c-8136-c56a7e28cbca service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:18.705900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262d2ec-f7fe-4a3c-8136-c56a7e28cbca service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:18.706170 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262d2ec-f7fe-4a3c-8136-c56a7e28cbca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:18.707748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6262d2ec-f7fe-4a3c-8136-c56a7e28cbca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:18.715544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:18.816207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262d2ec-f7fe-4a3c-8136-c56a7e28cbca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:18.817647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262d2ec-f7fe-4a3c-8136-c56a7e28cbca service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:18.835776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 656/1322] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:21.392909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016208e-84f8-42da-861e-8287c0c0c6de None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:21.393449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016208e-84f8-42da-861e-8287c0c0c6de None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:21.393449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016208e-84f8-42da-861e-8287c0c0c6de None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:21.394916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 667/1323] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:21.400461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d2f2c-e91d-435a-8e01-ce7a357e435d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:21.400904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d2f2c-e91d-435a-8e01-ce7a357e435d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:21.401085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d2f2c-e91d-435a-8e01-ce7a357e435d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:21.467698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5e4d2f2c-e91d-435a-8e01-ce7a357e435d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:21.468924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5e4d2f2c-e91d-435a-8e01-ce7a357e435d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:21.529971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:21.571744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 657/1324] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:21.854458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016208e-84f8-42da-861e-8287c0c0c6de None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:21.856522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c016208e-84f8-42da-861e-8287c0c0c6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:21.963510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016208e-84f8-42da-861e-8287c0c0c6de None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:21.975827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:22.038497 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee91fe4-1c7d-4280-99cd-9ef7000bb56b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:22.038986 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee91fe4-1c7d-4280-99cd-9ef7000bb56b service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:22.039263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee91fe4-1c7d-4280-99cd-9ef7000bb56b service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:22.039530 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee91fe4-1c7d-4280-99cd-9ef7000bb56b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:22.041128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-aee91fe4-1c7d-4280-99cd-9ef7000bb56b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:22.150297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee91fe4-1c7d-4280-99cd-9ef7000bb56b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:22.151772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee91fe4-1c7d-4280-99cd-9ef7000bb56b service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:22.170452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 668/1325] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:24.118260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ed149-5b91-464a-b2f8-beb81c995f55 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:24.118796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ed149-5b91-464a-b2f8-beb81c995f55 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:24.118796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ed149-5b91-464a-b2f8-beb81c995f55 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:24.120075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 658/1326] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:24.125546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28d085-9ef9-47ad-8a7a-bd3c60a05043 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:24.125955 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28d085-9ef9-47ad-8a7a-bd3c60a05043 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:24.126206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28d085-9ef9-47ad-8a7a-bd3c60a05043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:24.175297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:24.194693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8d28d085-9ef9-47ad-8a7a-bd3c60a05043 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:24.195886 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8d28d085-9ef9-47ad-8a7a-bd3c60a05043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:24.294620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 669/1327] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:24.572500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ed149-5b91-464a-b2f8-beb81c995f55 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:24.574053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-246ed149-5b91-464a-b2f8-beb81c995f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:24.678678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ed149-5b91-464a-b2f8-beb81c995f55 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:24.769359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd94cf7-3697-4f24-8df3-2f8e5a554e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:24.770147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd94cf7-3697-4f24-8df3-2f8e5a554e36 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:24.770230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd94cf7-3697-4f24-8df3-2f8e5a554e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:24.770481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd94cf7-3697-4f24-8df3-2f8e5a554e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:24.772106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-abd94cf7-3697-4f24-8df3-2f8e5a554e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:24.789752 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:24.885035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd94cf7-3697-4f24-8df3-2f8e5a554e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:24.886492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd94cf7-3697-4f24-8df3-2f8e5a554e36 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:24.904479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 659/1328] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:26.876929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228603d-86e7-4161-9ce4-5e7c9f68c3f2 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:26.877456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228603d-86e7-4161-9ce4-5e7c9f68c3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:26.877456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228603d-86e7-4161-9ce4-5e7c9f68c3f2 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:26.878802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 670/1329] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:26.884323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6c6fd-c5d7-48a2-bf86-e74f0f94c9a2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:26.884813 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6c6fd-c5d7-48a2-bf86-e74f0f94c9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:26.884912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6c6fd-c5d7-48a2-bf86-e74f0f94c9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:26.955842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b1f6c6fd-c5d7-48a2-bf86-e74f0f94c9a2 None None] MFA Rules not processed for user `7d7bdddb7df24e8a9615b6909130f4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:26.957091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b1f6c6fd-c5d7-48a2-bf86-e74f0f94c9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:27.023233 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:27.057086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 660/1330] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:27.344398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228603d-86e7-4161-9ce4-5e7c9f68c3f2 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:27.346019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e228603d-86e7-4161-9ce4-5e7c9f68c3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:27.454831 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228603d-86e7-4161-9ce4-5e7c9f68c3f2 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:27.476017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:27.538175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758a9c2-55d6-4bff-85f9-ba3e1c270ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:27.538458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a9c2-55d6-4bff-85f9-ba3e1c270ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:27.538731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a9c2-55d6-4bff-85f9-ba3e1c270ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:27.538996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a9c2-55d6-4bff-85f9-ba3e1c270ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:27.540621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6758a9c2-55d6-4bff-85f9-ba3e1c270ab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:27.656683 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758a9c2-55d6-4bff-85f9-ba3e1c270ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:27.658153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758a9c2-55d6-4bff-85f9-ba3e1c270ab6 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:27.678095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 671/1331] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:29.812878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbec68-796f-4487-93c0-0ee583f44864 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:29.813429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbec68-796f-4487-93c0-0ee583f44864 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:29.813429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbec68-796f-4487-93c0-0ee583f44864 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:29.814752 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 661/1332] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:29.819896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad871b-1c47-4cb2-9cb7-9d934b158360 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:29.820328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad871b-1c47-4cb2-9cb7-9d934b158360 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:29.820544 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad871b-1c47-4cb2-9cb7-9d934b158360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:29.883770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:29.886719 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-09ad871b-1c47-4cb2-9cb7-9d934b158360 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:29.888536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-09ad871b-1c47-4cb2-9cb7-9d934b158360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:29.985039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 672/1333] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:55:30.265610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbec68-796f-4487-93c0-0ee583f44864 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:30.267134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2ccbec68-796f-4487-93c0-0ee583f44864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:30.370274 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbec68-796f-4487-93c0-0ee583f44864 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:30.443381 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f468955-8532-4579-8eb4-2bb30b33ffc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:30.443981 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f468955-8532-4579-8eb4-2bb30b33ffc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:30.444288 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f468955-8532-4579-8eb4-2bb30b33ffc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:30.444568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f468955-8532-4579-8eb4-2bb30b33ffc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:30.446172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6f468955-8532-4579-8eb4-2bb30b33ffc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:30.473095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:30.553942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f468955-8532-4579-8eb4-2bb30b33ffc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:30.555295 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f468955-8532-4579-8eb4-2bb30b33ffc0 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:30.573678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 662/1334] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:32.537015 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9815de6-0f1b-4646-8fa7-3b7b26cf9b83 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:32.537867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9815de6-0f1b-4646-8fa7-3b7b26cf9b83 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:32.537867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9815de6-0f1b-4646-8fa7-3b7b26cf9b83 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:32.538894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 673/1335] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:32.544129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfef32f-8470-49b7-bd3e-6d0ca7d4fd3a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:32.544596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfef32f-8470-49b7-bd3e-6d0ca7d4fd3a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:32.544693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfef32f-8470-49b7-bd3e-6d0ca7d4fd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:32.609574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-edfef32f-8470-49b7-bd3e-6d0ca7d4fd3a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:32.610759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-edfef32f-8470-49b7-bd3e-6d0ca7d4fd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:32.698597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:32.731279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 663/1336] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:33.014326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9815de6-0f1b-4646-8fa7-3b7b26cf9b83 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:33.015873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f9815de6-0f1b-4646-8fa7-3b7b26cf9b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:33.117805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9815de6-0f1b-4646-8fa7-3b7b26cf9b83 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:33.145996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:33.192237 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af4514-eb6e-46f9-946a-6dbe0823f4e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:33.192838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4514-eb6e-46f9-946a-6dbe0823f4e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:33.193119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4514-eb6e-46f9-946a-6dbe0823f4e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:33.193391 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4514-eb6e-46f9-946a-6dbe0823f4e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:33.194980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e4af4514-eb6e-46f9-946a-6dbe0823f4e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:33.303220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af4514-eb6e-46f9-946a-6dbe0823f4e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:33.304615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af4514-eb6e-46f9-946a-6dbe0823f4e3 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:33.327513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 674/1337] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:35.814840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4ef9-3993-453e-807c-566607397b52 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:35.815439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4ef9-3993-453e-807c-566607397b52 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:35.815439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4ef9-3993-453e-807c-566607397b52 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:35.816778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 664/1338] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:35.821934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f90d9e-85b2-471f-b7e0-da4d9dd09cc3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:35.822356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f90d9e-85b2-471f-b7e0-da4d9dd09cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:35.822489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f90d9e-85b2-471f-b7e0-da4d9dd09cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:35.887403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c6f90d9e-85b2-471f-b7e0-da4d9dd09cc3 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:35.888627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c6f90d9e-85b2-471f-b7e0-da4d9dd09cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:35.897400 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:35.984828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 675/1339] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:36.271503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a4ef9-3993-453e-807c-566607397b52 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:36.273059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab0a4ef9-3993-453e-807c-566607397b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:36.383226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a4ef9-3993-453e-807c-566607397b52 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:36.456193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a8dba-37e0-4e1c-a346-b88e383d4791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:36.456790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a8dba-37e0-4e1c-a346-b88e383d4791 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:36.457075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a8dba-37e0-4e1c-a346-b88e383d4791 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:36.457347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a8dba-37e0-4e1c-a346-b88e383d4791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:36.459042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b29a8dba-37e0-4e1c-a346-b88e383d4791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:36.494723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:36.567445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a8dba-37e0-4e1c-a346-b88e383d4791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:36.568845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a8dba-37e0-4e1c-a346-b88e383d4791 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:36.586641 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 665/1340] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:40.416532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5e20c-51dd-4542-8f06-5b54faa6fa1e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:40.417040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5e20c-51dd-4542-8f06-5b54faa6fa1e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:40.417140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5e20c-51dd-4542-8f06-5b54faa6fa1e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:40.418417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 676/1341] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:40.424685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6df4-464c-459c-a138-4b59ddeac77c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:40.425137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6df4-464c-459c-a138-4b59ddeac77c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:40.425251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6df4-464c-459c-a138-4b59ddeac77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:40.494879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6b6b6df4-464c-459c-a138-4b59ddeac77c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:40.496111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6b6b6df4-464c-459c-a138-4b59ddeac77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:40.581089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:40.593785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 666/1342] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:40.872595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5e20c-51dd-4542-8f06-5b54faa6fa1e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:40.874163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b7c5e20c-51dd-4542-8f06-5b54faa6fa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:40.978298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5e20c-51dd-4542-8f06-5b54faa6fa1e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:41.017085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:41.053545 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b91f7-f023-4320-b100-25936670abd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:41.054057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b91f7-f023-4320-b100-25936670abd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:41.054332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b91f7-f023-4320-b100-25936670abd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:41.054600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b91f7-f023-4320-b100-25936670abd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:41.056197 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a63b91f7-f023-4320-b100-25936670abd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:41.163697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b91f7-f023-4320-b100-25936670abd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:41.165175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b91f7-f023-4320-b100-25936670abd2 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:41.182963 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 677/1343] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:43.670078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360a451-6612-4c09-bec2-9ef2373fe545 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:43.670700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360a451-6612-4c09-bec2-9ef2373fe545 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:43.670700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360a451-6612-4c09-bec2-9ef2373fe545 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:43.672196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 667/1344] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:43.678058 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503cde1-affc-40cb-abb9-0ac113c847c9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:43.678571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503cde1-affc-40cb-abb9-0ac113c847c9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:43.678662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503cde1-affc-40cb-abb9-0ac113c847c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:43.754661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6503cde1-affc-40cb-abb9-0ac113c847c9 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:43.755810 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6503cde1-affc-40cb-abb9-0ac113c847c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:43.778880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:43.880911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 678/1345] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:44.160772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360a451-6612-4c09-bec2-9ef2373fe545 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:44.162296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1360a451-6612-4c09-bec2-9ef2373fe545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:44.268624 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360a451-6612-4c09-bec2-9ef2373fe545 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:44.345524 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586accc-bef5-4b51-8f90-efe45d64c103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:44.346126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586accc-bef5-4b51-8f90-efe45d64c103 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:44.346405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586accc-bef5-4b51-8f90-efe45d64c103 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:44.346675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586accc-bef5-4b51-8f90-efe45d64c103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:44.348286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1586accc-bef5-4b51-8f90-efe45d64c103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:44.394182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:44.464979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586accc-bef5-4b51-8f90-efe45d64c103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:44.466407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586accc-bef5-4b51-8f90-efe45d64c103 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:44.484646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 668/1346] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:48.463581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191e624-c0ba-4aa3-85b0-9aac769494b5 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:48.464205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191e624-c0ba-4aa3-85b0-9aac769494b5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:48.464205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191e624-c0ba-4aa3-85b0-9aac769494b5 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:48.465669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 679/1347] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:48.471615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a40528-1756-49e6-abd8-64aca7598751 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:48.472294 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a40528-1756-49e6-abd8-64aca7598751 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:48.472294 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a40528-1756-49e6-abd8-64aca7598751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:48.560601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-08a40528-1756-49e6-abd8-64aca7598751 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:48.561914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-08a40528-1756-49e6-abd8-64aca7598751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:48.663527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:48.666527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 669/1348] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:48.951847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191e624-c0ba-4aa3-85b0-9aac769494b5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:48.953563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a191e624-c0ba-4aa3-85b0-9aac769494b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:49.061402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191e624-c0ba-4aa3-85b0-9aac769494b5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:49.111498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:49.139825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11699889-5bc1-4f6c-aab1-f3e287d6b4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:49.140319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11699889-5bc1-4f6c-aab1-f3e287d6b4aa service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:49.140600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11699889-5bc1-4f6c-aab1-f3e287d6b4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:49.140872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11699889-5bc1-4f6c-aab1-f3e287d6b4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:49.142460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-11699889-5bc1-4f6c-aab1-f3e287d6b4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:49.251846 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11699889-5bc1-4f6c-aab1-f3e287d6b4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:49.253422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11699889-5bc1-4f6c-aab1-f3e287d6b4aa service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:49.271376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 680/1349] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:54.237239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab1fba-7005-43e5-b615-68db8981090c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:54.237652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab1fba-7005-43e5-b615-68db8981090c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:54.239258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab1fba-7005-43e5-b615-68db8981090c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:54.240329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 670/1350] 10.222.0.86 () {54 vars in 921 bytes} [Wed Aug 24 09:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:54.244642 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b0c1-465c-4b42-a512-715ae5ff87a5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:54.245072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b0c1-465c-4b42-a512-715ae5ff87a5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:54.245252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b0c1-465c-4b42-a512-715ae5ff87a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:54.353957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fcc4b0c1-465c-4b42-a512-715ae5ff87a5 None None] MFA Rules not processed for user `ef8d30e87c624d6180656d998afcebe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:54.355246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fcc4b0c1-465c-4b42-a512-715ae5ff87a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:54.377166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:54.493796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 681/1351] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Aug 24 09:55:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:58.890740 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5db68-9282-4ebd-bc09-dca5fabf673a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:58.891280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5db68-9282-4ebd-bc09-dca5fabf673a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:58.891363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5db68-9282-4ebd-bc09-dca5fabf673a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:58.892850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 671/1352] 10.222.0.86 () {54 vars in 921 bytes} [Wed Aug 24 09:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:58.897749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73530852-41bf-447a-82bc-e7abacdfe9b0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:58.898881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73530852-41bf-447a-82bc-e7abacdfe9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:58.899390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73530852-41bf-447a-82bc-e7abacdfe9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:58.979953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-73530852-41bf-447a-82bc-e7abacdfe9b0 None None] MFA Rules not processed for user `ef8d30e87c624d6180656d998afcebe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:58.981184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-73530852-41bf-447a-82bc-e7abacdfe9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:59.059091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:59.098929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 682/1353] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Aug 24 09:55:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:00.502002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe81b27-85e3-4637-9a90-d54bb0c1835c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:00.502534 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe81b27-85e3-4637-9a90-d54bb0c1835c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:00.502534 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe81b27-85e3-4637-9a90-d54bb0c1835c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:00.504087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 672/1354] 10.222.0.86 () {54 vars in 921 bytes} [Wed Aug 24 09:56:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:00.509772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7238ea-80c9-4361-9c84-c14db824b927 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:00.510300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7238ea-80c9-4361-9c84-c14db824b927 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:00.510428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7238ea-80c9-4361-9c84-c14db824b927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:00.588231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1b7238ea-80c9-4361-9c84-c14db824b927 None None] MFA Rules not processed for user `ef8d30e87c624d6180656d998afcebe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:00.589435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1b7238ea-80c9-4361-9c84-c14db824b927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:00.708892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 683/1355] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Aug 24 09:56:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:56:03.927157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cf5dc-01ff-4c93-879a-efcb7e046d72 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:03.933942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cf5dc-01ff-4c93-879a-efcb7e046d72 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:03.934496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cf5dc-01ff-4c93-879a-efcb7e046d72 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:03.937091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 673/1356] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:03.943555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369ad3-926d-49be-a42c-0a3f62f49ed2 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:03.944063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369ad3-926d-49be-a42c-0a3f62f49ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:03.944177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369ad3-926d-49be-a42c-0a3f62f49ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:04.001121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:04.049336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-53369ad3-926d-49be-a42c-0a3f62f49ed2 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:04.050445 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-53369ad3-926d-49be-a42c-0a3f62f49ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:04.216770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 684/1357] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:04.519203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01135b3-2db3-4102-a3a6-2b68fb1679d3 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:04.519880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01135b3-2db3-4102-a3a6-2b68fb1679d3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:04.520398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01135b3-2db3-4102-a3a6-2b68fb1679d3 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:04.522508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 674/1358] 10.222.0.86 () {54 vars in 962 bytes} [Wed Aug 24 09:56:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:04.530193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11bb0d-0e42-45a3-b709-2a412440c636 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:04.530471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11bb0d-0e42-45a3-b709-2a412440c636 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:04.530770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11bb0d-0e42-45a3-b709-2a412440c636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:04.618563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7c11bb0d-0e42-45a3-b709-2a412440c636 None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:04.619755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7c11bb0d-0e42-45a3-b709-2a412440c636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:04.692631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:04.773477 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 685/1359] 10.222.0.86 () {58 vars in 1059 bytes} [Wed Aug 24 09:56:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:04.783658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01135b3-2db3-4102-a3a6-2b68fb1679d3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:04.787514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a01135b3-2db3-4102-a3a6-2b68fb1679d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:04.942842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01135b3-2db3-4102-a3a6-2b68fb1679d3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:05.019071 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:05.060721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06903c-966f-4136-84f3-dc5b8a40f85e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:05.061229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06903c-966f-4136-84f3-dc5b8a40f85e service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:05.061503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06903c-966f-4136-84f3-dc5b8a40f85e service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:05.061775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06903c-966f-4136-84f3-dc5b8a40f85e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:05.063882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1e06903c-966f-4136-84f3-dc5b8a40f85e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:05.158115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06903c-966f-4136-84f3-dc5b8a40f85e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:05.159719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06903c-966f-4136-84f3-dc5b8a40f85e service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:05.178638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 675/1360] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:07.359529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b894-eb8d-495a-bf66-dfeec12e8937 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:07.360138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b894-eb8d-495a-bf66-dfeec12e8937 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:07.360138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b894-eb8d-495a-bf66-dfeec12e8937 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:07.361899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 686/1361] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:07.367515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2a2a6-9ad2-4663-a093-9feb82274090 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:07.367515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2a2a6-9ad2-4663-a093-9feb82274090 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:07.368131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2a2a6-9ad2-4663-a093-9feb82274090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:07.441356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-edf2a2a6-9ad2-4663-a093-9feb82274090 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:07.442626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-edf2a2a6-9ad2-4663-a093-9feb82274090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:07.496102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:07.575622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 676/1362] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:07.870650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6b894-eb8d-495a-bf66-dfeec12e8937 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:07.872382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d1c6b894-eb8d-495a-bf66-dfeec12e8937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:08.035946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6b894-eb8d-495a-bf66-dfeec12e8937 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:08.049099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:08.182354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fc7c2-39c4-492f-ad0e-6b85488999a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:08.182939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fc7c2-39c4-492f-ad0e-6b85488999a0 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:08.183220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fc7c2-39c4-492f-ad0e-6b85488999a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:08.183492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fc7c2-39c4-492f-ad0e-6b85488999a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:08.185120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a53fc7c2-39c4-492f-ad0e-6b85488999a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:08.293620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53fc7c2-39c4-492f-ad0e-6b85488999a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:08.295568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53fc7c2-39c4-492f-ad0e-6b85488999a0 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:08.315810 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 687/1363] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:10.596679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14431278-1f5f-498b-974d-fb92c1b7925d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:10.597221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14431278-1f5f-498b-974d-fb92c1b7925d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:10.597221 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14431278-1f5f-498b-974d-fb92c1b7925d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:10.598590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 677/1364] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:10.604133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5d2ff-0999-4dd5-b12a-b024d5ea1cba None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:10.604591 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5d2ff-0999-4dd5-b12a-b024d5ea1cba None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:10.604884 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5d2ff-0999-4dd5-b12a-b024d5ea1cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:10.621185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:10.679257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-12e5d2ff-0999-4dd5-b12a-b024d5ea1cba None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:10.680648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-12e5d2ff-0999-4dd5-b12a-b024d5ea1cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:10.800088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 688/1365] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:11.098158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd4c09-4198-4896-8bb2-37c3ee702644 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:11.098598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd4c09-4198-4896-8bb2-37c3ee702644 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:11.098977 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd4c09-4198-4896-8bb2-37c3ee702644 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:11.100597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 678/1366] 10.222.0.86 () {54 vars in 962 bytes} [Wed Aug 24 09:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:11.106650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb761e19-124f-44da-a56c-3ad620db46a2 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:11.106937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb761e19-124f-44da-a56c-3ad620db46a2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:11.107217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb761e19-124f-44da-a56c-3ad620db46a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:11.193940 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-bb761e19-124f-44da-a56c-3ad620db46a2 None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:11.195954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bb761e19-124f-44da-a56c-3ad620db46a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:11.252580 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:11.357957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 689/1367] 10.222.0.86 () {58 vars in 1059 bytes} [Wed Aug 24 09:56:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:11.359055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd4c09-4198-4896-8bb2-37c3ee702644 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:11.361983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-00bd4c09-4198-4896-8bb2-37c3ee702644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:11.516721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd4c09-4198-4896-8bb2-37c3ee702644 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:11.544611 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:11.650200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cda78-dc11-4912-a1eb-e199beb5b63c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:11.650881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cda78-dc11-4912-a1eb-e199beb5b63c service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:11.651191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cda78-dc11-4912-a1eb-e199beb5b63c service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:11.651366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cda78-dc11-4912-a1eb-e199beb5b63c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:11.653072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab1cda78-dc11-4912-a1eb-e199beb5b63c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:11.749144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cda78-dc11-4912-a1eb-e199beb5b63c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:11.750721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cda78-dc11-4912-a1eb-e199beb5b63c service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:11.770350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 679/1368] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:56:12.135429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88b738-d503-4143-b473-3fea6e5a8aaf None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:12.135940 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88b738-d503-4143-b473-3fea6e5a8aaf None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:12.135940 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88b738-d503-4143-b473-3fea6e5a8aaf None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:12.137296 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 690/1369] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:12.142582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afd955-70a6-4c87-a0eb-5b52021b5b6a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:12.142874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afd955-70a6-4c87-a0eb-5b52021b5b6a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:12.143154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afd955-70a6-4c87-a0eb-5b52021b5b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:12.177329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:12.216985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-77afd955-70a6-4c87-a0eb-5b52021b5b6a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:12.218353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-77afd955-70a6-4c87-a0eb-5b52021b5b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:12.327044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 680/1370] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:12.925538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79299daa-6a33-4a5a-b6c8-d84ba3a36738 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:12.926087 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79299daa-6a33-4a5a-b6c8-d84ba3a36738 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:12.926536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79299daa-6a33-4a5a-b6c8-d84ba3a36738 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:12.928342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 691/1371] 10.222.0.86 () {54 vars in 964 bytes} [Wed Aug 24 09:56:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:12.933223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59a827-4257-444b-ab6e-f350f7f93e79 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:12.934350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59a827-4257-444b-ab6e-f350f7f93e79 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:12.934855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59a827-4257-444b-ab6e-f350f7f93e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:13.020512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7b59a827-4257-444b-ab6e-f350f7f93e79 None None] MFA Rules not processed for user `bd7b580705f247699aeb58772e668730`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:13.021666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7b59a827-4257-444b-ab6e-f350f7f93e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:13.074684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:13.106850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 681/1372] 10.222.0.86 () {58 vars in 1061 bytes} [Wed Aug 24 09:56:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:56:13.114610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79299daa-6a33-4a5a-b6c8-d84ba3a36738 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:13.116849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-79299daa-6a33-4a5a-b6c8-d84ba3a36738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:13.229027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79299daa-6a33-4a5a-b6c8-d84ba3a36738 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:13.279346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:13.309059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d83ad-bb79-4795-ba3e-158a39b90df6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:13.309707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d83ad-bb79-4795-ba3e-158a39b90df6 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:13.309984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d83ad-bb79-4795-ba3e-158a39b90df6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:13.310253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d83ad-bb79-4795-ba3e-158a39b90df6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:13.311847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2a9d83ad-bb79-4795-ba3e-158a39b90df6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:13.430665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d83ad-bb79-4795-ba3e-158a39b90df6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:13.432151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d83ad-bb79-4795-ba3e-158a39b90df6 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:13.452700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 692/1373] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:13.823179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d06b6f-9a79-465d-8908-b28b55217ec9 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:13.823369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d06b6f-9a79-465d-8908-b28b55217ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:13.823648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d06b6f-9a79-465d-8908-b28b55217ec9 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:13.825070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 682/1374] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:13.830406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344da2-4500-4fa9-a98b-82e3d00bd010 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:13.830771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344da2-4500-4fa9-a98b-82e3d00bd010 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:13.831051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344da2-4500-4fa9-a98b-82e3d00bd010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:13.899283 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c3344da2-4500-4fa9-a98b-82e3d00bd010 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:13.900564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c3344da2-4500-4fa9-a98b-82e3d00bd010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:13.924414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:14.011174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 693/1375] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:14.305340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d06b6f-9a79-465d-8908-b28b55217ec9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:14.306856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a6d06b6f-9a79-465d-8908-b28b55217ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:14.456759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d06b6f-9a79-465d-8908-b28b55217ec9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:14.508552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:14.576297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953b128-1b1b-4036-8c9a-85c571e98dd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:14.576898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953b128-1b1b-4036-8c9a-85c571e98dd6 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:14.577173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953b128-1b1b-4036-8c9a-85c571e98dd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:14.577443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953b128-1b1b-4036-8c9a-85c571e98dd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:14.579105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e953b128-1b1b-4036-8c9a-85c571e98dd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:14.671881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953b128-1b1b-4036-8c9a-85c571e98dd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:14.674199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953b128-1b1b-4036-8c9a-85c571e98dd6 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:14.693188 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 683/1376] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:16.647806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e5f50-c05b-4f40-9f7e-c23e15b82bad None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:16.648366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e5f50-c05b-4f40-9f7e-c23e15b82bad None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:16.648366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e5f50-c05b-4f40-9f7e-c23e15b82bad None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:16.649650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 694/1377] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:16.655282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac04687-8919-4df9-8ecd-8009462b3b4d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:16.655712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac04687-8919-4df9-8ecd-8009462b3b4d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:16.655837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac04687-8919-4df9-8ecd-8009462b3b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:16.722365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3ac04687-8919-4df9-8ecd-8009462b3b4d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:16.723578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3ac04687-8919-4df9-8ecd-8009462b3b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:16.730629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:16.833723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 684/1378] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:17.131242 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e5f50-c05b-4f40-9f7e-c23e15b82bad None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:17.132882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6d9e5f50-c05b-4f40-9f7e-c23e15b82bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:17.297540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e5f50-c05b-4f40-9f7e-c23e15b82bad None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:17.301831 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:17.428876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07140c5-9608-465d-8381-bb38d49effed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:17.429692 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07140c5-9608-465d-8381-bb38d49effed service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:17.430070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07140c5-9608-465d-8381-bb38d49effed service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:17.430070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07140c5-9608-465d-8381-bb38d49effed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:17.431673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a07140c5-9608-465d-8381-bb38d49effed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:17.533723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07140c5-9608-465d-8381-bb38d49effed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:17.535438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07140c5-9608-465d-8381-bb38d49effed service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:17.557121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 695/1379] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:19.699119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1970a6-0c31-46b8-b098-f7282af6dede None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:19.699661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1970a6-0c31-46b8-b098-f7282af6dede None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:19.700003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1970a6-0c31-46b8-b098-f7282af6dede None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:19.701966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 685/1380] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:19.708786 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88b0f-ee78-4b0f-938b-d2707148bc0d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:19.709274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88b0f-ee78-4b0f-938b-d2707148bc0d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:19.709360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88b0f-ee78-4b0f-938b-d2707148bc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:19.721503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:19.786279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d5b88b0f-ee78-4b0f-938b-d2707148bc0d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:19.786279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d5b88b0f-ee78-4b0f-938b-d2707148bc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:19.920569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 696/1381] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:20.229039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1970a6-0c31-46b8-b098-f7282af6dede None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:20.230702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-da1970a6-0c31-46b8-b098-f7282af6dede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:20.385255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:20.397558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1970a6-0c31-46b8-b098-f7282af6dede None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:20.531195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaf199-5ff9-47f9-949a-8cd50caf9ed6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:20.531765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaf199-5ff9-47f9-949a-8cd50caf9ed6 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:20.532032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaf199-5ff9-47f9-949a-8cd50caf9ed6 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:20.532330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaf199-5ff9-47f9-949a-8cd50caf9ed6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:20.533919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d7aaf199-5ff9-47f9-949a-8cd50caf9ed6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:20.633834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaf199-5ff9-47f9-949a-8cd50caf9ed6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:20.635447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaf199-5ff9-47f9-949a-8cd50caf9ed6 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:20.647005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:20.656352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 686/1382] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:23.054640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b695a-ef54-4b6b-98bc-1ee0fca93c34 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:23.055422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b695a-ef54-4b6b-98bc-1ee0fca93c34 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:23.055422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b695a-ef54-4b6b-98bc-1ee0fca93c34 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:23.056509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 697/1383] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:23.063391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbfb58-e13a-42b5-8c27-b335d36a0427 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:23.063973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbfb58-e13a-42b5-8c27-b335d36a0427 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:23.063973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbfb58-e13a-42b5-8c27-b335d36a0427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:23.131693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a6bbfb58-e13a-42b5-8c27-b335d36a0427 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:23.132974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a6bbfb58-e13a-42b5-8c27-b335d36a0427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:23.230306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 687/1384] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:23.607567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b695a-ef54-4b6b-98bc-1ee0fca93c34 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:23.609223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2b2b695a-ef54-4b6b-98bc-1ee0fca93c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:23.709339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:23.775109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b695a-ef54-4b6b-98bc-1ee0fca93c34 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:23.905783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07db55f-29cf-45f7-98f7-c0be9ae45526 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:23.907124 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db55f-29cf-45f7-98f7-c0be9ae45526 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:23.907554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db55f-29cf-45f7-98f7-c0be9ae45526 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:23.907896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db55f-29cf-45f7-98f7-c0be9ae45526 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:23.910048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c07db55f-29cf-45f7-98f7-c0be9ae45526 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:23.964650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:24.006888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07db55f-29cf-45f7-98f7-c0be9ae45526 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:24.008692 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07db55f-29cf-45f7-98f7-c0be9ae45526 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:24.028122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 698/1385] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:25.998091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba66ad2-62d0-40bc-b9f4-611cb633c606 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:25.998639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba66ad2-62d0-40bc-b9f4-611cb633c606 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:25.998801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba66ad2-62d0-40bc-b9f4-611cb633c606 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:26.000714 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 688/1386] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:26.006130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6555-5479-4a09-af40-a9b4ca789d1e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:26.006567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6555-5479-4a09-af40-a9b4ca789d1e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:26.006698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6555-5479-4a09-af40-a9b4ca789d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:26.080237 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1bfe6555-5479-4a09-af40-a9b4ca789d1e None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:26.081389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1bfe6555-5479-4a09-af40-a9b4ca789d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:26.182844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 699/1387] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:26.556795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba66ad2-62d0-40bc-b9f4-611cb633c606 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:26.558426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2ba66ad2-62d0-40bc-b9f4-611cb633c606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:26.640825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:26.716463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba66ad2-62d0-40bc-b9f4-611cb633c606 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:26.845177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee94ed7-92ea-4123-83bc-41309225ada4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:26.845701 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee94ed7-92ea-4123-83bc-41309225ada4 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:26.845975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee94ed7-92ea-4123-83bc-41309225ada4 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:26.846246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee94ed7-92ea-4123-83bc-41309225ada4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:26.848120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8ee94ed7-92ea-4123-83bc-41309225ada4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:26.892965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:26.946579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee94ed7-92ea-4123-83bc-41309225ada4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:26.948224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee94ed7-92ea-4123-83bc-41309225ada4 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:26.966861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 689/1388] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:28.511101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c581f3-8ead-428e-a7f0-abf1ba946993 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:28.511639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c581f3-8ead-428e-a7f0-abf1ba946993 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:28.511732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c581f3-8ead-428e-a7f0-abf1ba946993 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:28.513219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 700/1389] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:28.519210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b15be-5a46-4a92-9af2-2ac0c4f43634 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:28.519673 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b15be-5a46-4a92-9af2-2ac0c4f43634 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:28.519892 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b15be-5a46-4a92-9af2-2ac0c4f43634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:28.589190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-717b15be-5a46-4a92-9af2-2ac0c4f43634 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:28.590442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-717b15be-5a46-4a92-9af2-2ac0c4f43634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:28.690487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 690/1390] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:28.919576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10649104-1bd3-4a5d-9fd7-75fa5848265f None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:28.919852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10649104-1bd3-4a5d-9fd7-75fa5848265f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:28.920152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10649104-1bd3-4a5d-9fd7-75fa5848265f None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:28.921648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 701/1391] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:28.926791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944123a-c7a6-44e5-b5f6-f8c5cd0c82dc None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:28.927329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944123a-c7a6-44e5-b5f6-f8c5cd0c82dc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:28.927329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944123a-c7a6-44e5-b5f6-f8c5cd0c82dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:28.943645 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:28.988639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7498e9-d3f1-41d4-aaa0-6c8e74607f33 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:28.988922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7498e9-d3f1-41d4-aaa0-6c8e74607f33 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:28.989196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7498e9-d3f1-41d4-aaa0-6c8e74607f33 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:28.990537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 702/1392] 10.222.0.86 () {54 vars in 967 bytes} [Wed Aug 24 09:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:56:28.995347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e64bb2-c4be-47ab-b032-2315876791de None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:28.995625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e64bb2-c4be-47ab-b032-2315876791de None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:28.995894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e64bb2-c4be-47ab-b032-2315876791de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:29.001422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f944123a-c7a6-44e5-b5f6-f8c5cd0c82dc None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:29.002675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f944123a-c7a6-44e5-b5f6-f8c5cd0c82dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:29.029067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:29.079250 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-08e64bb2-c4be-47ab-b032-2315876791de None None] MFA Rules not processed for user `794dbf38682e4b4f939854eecbacaf07`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:29.080505 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-08e64bb2-c4be-47ab-b032-2315876791de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:29.102608 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 691/1393] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:29.156760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 703/1394] 10.222.0.86 () {58 vars in 1064 bytes} [Wed Aug 24 09:56:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:56:29.164335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944123a-c7a6-44e5-b5f6-f8c5cd0c82dc None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:29.165530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f944123a-c7a6-44e5-b5f6-f8c5cd0c82dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:29.256763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944123a-c7a6-44e5-b5f6-f8c5cd0c82dc None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:29.265183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:29.332515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b0b80-e507-4381-9003-967b6fbdb0e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '794dbf38682e4b4f939854eecbacaf07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:29.333029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0b80-e507-4381-9003-967b6fbdb0e8 service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:29.333308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0b80-e507-4381-9003-967b6fbdb0e8 service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:29.333583 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0b80-e507-4381-9003-967b6fbdb0e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:29.335165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-301b0b80-e507-4381-9003-967b6fbdb0e8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:29.391395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e64bb2-c4be-47ab-b032-2315876791de None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:29.392636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-08e64bb2-c4be-47ab-b032-2315876791de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:29.449996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b0b80-e507-4381-9003-967b6fbdb0e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:29.451504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b0b80-e507-4381-9003-967b6fbdb0e8 service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:29.470828 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 692/1395] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:29.513594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e64bb2-c4be-47ab-b032-2315876791de None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:29.517201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:29.627754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67b1b0-cbc9-4bf4-8561-f6b4d81e14ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:29.628397 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67b1b0-cbc9-4bf4-8561-f6b4d81e14ee service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:29.628680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67b1b0-cbc9-4bf4-8561-f6b4d81e14ee service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:29.628958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67b1b0-cbc9-4bf4-8561-f6b4d81e14ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:29.630614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bb67b1b0-cbc9-4bf4-8561-f6b4d81e14ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:29.727856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67b1b0-cbc9-4bf4-8561-f6b4d81e14ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:29.729529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67b1b0-cbc9-4bf4-8561-f6b4d81e14ee service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:29.750902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 704/1396] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:56:30.221566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeffcb-7656-4c20-a34c-4343a367926d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:30.222080 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeffcb-7656-4c20-a34c-4343a367926d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:30.222240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeffcb-7656-4c20-a34c-4343a367926d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:30.268841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:30.291767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-96eeffcb-7656-4c20-a34c-4343a367926d None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:30.292998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-96eeffcb-7656-4c20-a34c-4343a367926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:30.379438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 693/1397] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 09:56:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:30.425924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265fcba-386c-4900-b946-96132e7d873c None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:30.426305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265fcba-386c-4900-b946-96132e7d873c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:30.426758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265fcba-386c-4900-b946-96132e7d873c None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:30.428231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 705/1398] 10.222.0.86 () {54 vars in 943 bytes} [Wed Aug 24 09:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:30.435259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a7180-d4da-4f1d-9474-8acfc8717022 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:30.435613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a7180-d4da-4f1d-9474-8acfc8717022 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:30.435954 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a7180-d4da-4f1d-9474-8acfc8717022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:30.512413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2e3a7180-d4da-4f1d-9474-8acfc8717022 None None] MFA Rules not processed for user `f1b1d85195db46b19d6a0156dbdf7f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:30.513525 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2e3a7180-d4da-4f1d-9474-8acfc8717022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:30.574234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:30.619546 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 694/1399] 10.222.0.86 () {58 vars in 1040 bytes} [Wed Aug 24 09:56:30 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:30.629473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265fcba-386c-4900-b946-96132e7d873c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:30.630982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b265fcba-386c-4900-b946-96132e7d873c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:30.641559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:30.777925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265fcba-386c-4900-b946-96132e7d873c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:30.883586 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:30.890895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9407361-70cb-4aea-a5c6-539c47f97c4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:30.891504 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9407361-70cb-4aea-a5c6-539c47f97c4a service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:30.891784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9407361-70cb-4aea-a5c6-539c47f97c4a service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:30.892079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9407361-70cb-4aea-a5c6-539c47f97c4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:30.893735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f9407361-70cb-4aea-a5c6-539c47f97c4a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:30.976548 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9407361-70cb-4aea-a5c6-539c47f97c4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:30.977837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9407361-70cb-4aea-a5c6-539c47f97c4a service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:30.986204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 706/1400] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 09:56:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:56:31.715924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aad334-53ed-4d5f-bc54-b95c385e8ac7 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:31.716493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aad334-53ed-4d5f-bc54-b95c385e8ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:31.716493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aad334-53ed-4d5f-bc54-b95c385e8ac7 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:31.719765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 695/1401] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:31.723610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093e4a-5d21-49b7-9f89-68a0f26cb61d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:31.724125 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093e4a-5d21-49b7-9f89-68a0f26cb61d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:31.724225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093e4a-5d21-49b7-9f89-68a0f26cb61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:31.817499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1c093e4a-5d21-49b7-9f89-68a0f26cb61d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:31.818736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1c093e4a-5d21-49b7-9f89-68a0f26cb61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:31.910034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:31.935316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 707/1402] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:32.263212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aad334-53ed-4d5f-bc54-b95c385e8ac7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:32.265724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-41aad334-53ed-4d5f-bc54-b95c385e8ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:32.444006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aad334-53ed-4d5f-bc54-b95c385e8ac7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:32.491064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:32.592744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284448b4-9f90-4539-a680-492fc865e736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:32.593164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284448b4-9f90-4539-a680-492fc865e736 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:32.593475 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284448b4-9f90-4539-a680-492fc865e736 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:32.593657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284448b4-9f90-4539-a680-492fc865e736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:32.597317 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-284448b4-9f90-4539-a680-492fc865e736 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:32.716581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284448b4-9f90-4539-a680-492fc865e736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:32.718224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284448b4-9f90-4539-a680-492fc865e736 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:32.738817 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 696/1403] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:34.719624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62a0f9-54e8-474b-a400-8083d44fb0e4 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.720208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62a0f9-54e8-474b-a400-8083d44fb0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.720208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62a0f9-54e8-474b-a400-8083d44fb0e4 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:34.721541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 708/1404] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:34.727101 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b5ac6-6321-45e4-92fd-eacde0f1343f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:34.727899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b5ac6-6321-45e4-92fd-eacde0f1343f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:34.728704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b5ac6-6321-45e4-92fd-eacde0f1343f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:34.796126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:34.811807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-156b5ac6-6321-45e4-92fd-eacde0f1343f None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:34.813711 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-156b5ac6-6321-45e4-92fd-eacde0f1343f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:34.920102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 697/1405] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:35.214519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62a0f9-54e8-474b-a400-8083d44fb0e4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:35.216169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9c62a0f9-54e8-474b-a400-8083d44fb0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:35.391583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62a0f9-54e8-474b-a400-8083d44fb0e4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:35.455928 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:35.514113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d42f8-4c3f-405d-b415-84ceb8adcc3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:35.514715 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d42f8-4c3f-405d-b415-84ceb8adcc3d service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:35.514992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d42f8-4c3f-405d-b415-84ceb8adcc3d service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:35.515259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d42f8-4c3f-405d-b415-84ceb8adcc3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:35.516933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3f2d42f8-4c3f-405d-b415-84ceb8adcc3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:35.620119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d42f8-4c3f-405d-b415-84ceb8adcc3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:35.621744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d42f8-4c3f-405d-b415-84ceb8adcc3d service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:35.648448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 709/1406] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:37.635815 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c739ac-a0e3-4a1a-9fa8-0739dbc3f182 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:37.636415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c739ac-a0e3-4a1a-9fa8-0739dbc3f182 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:37.636415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c739ac-a0e3-4a1a-9fa8-0739dbc3f182 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:37.637748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 698/1407] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:37.643700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e16bc-cb25-4534-bdfc-5a3e4a797798 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:37.644561 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e16bc-cb25-4534-bdfc-5a3e4a797798 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:37.645002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e16bc-cb25-4534-bdfc-5a3e4a797798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:37.801309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2d9e16bc-cb25-4534-bdfc-5a3e4a797798 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:37.802868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2d9e16bc-cb25-4534-bdfc-5a3e4a797798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:37.828425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:37.919166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 710/1408] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:38.216299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c739ac-a0e3-4a1a-9fa8-0739dbc3f182 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:38.217833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-61c739ac-a0e3-4a1a-9fa8-0739dbc3f182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:38.367939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:38.397299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c739ac-a0e3-4a1a-9fa8-0739dbc3f182 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:38.528273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a5f46-95b6-4616-8b9c-976457e27898 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:38.528949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a5f46-95b6-4616-8b9c-976457e27898 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:38.529108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a5f46-95b6-4616-8b9c-976457e27898 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:38.529377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a5f46-95b6-4616-8b9c-976457e27898 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:38.531028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-093a5f46-95b6-4616-8b9c-976457e27898 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:38.628300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:38.632930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a5f46-95b6-4616-8b9c-976457e27898 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:38.634944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a5f46-95b6-4616-8b9c-976457e27898 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:38.656082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 699/1409] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:47.915446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=33 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:48.652154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:53.361355 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a717-1beb-490c-9415-f36a88007ae6 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.362188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a717-1beb-490c-9415-f36a88007ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.362188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a717-1beb-490c-9415-f36a88007ae6 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.363577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 711/1410] 10.222.0.86 () {54 vars in 918 bytes} [Wed Aug 24 09:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:53.368584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d47ac-2fab-4e49-bcbf-f9e75568a728 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.369002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d47ac-2fab-4e49-bcbf-f9e75568a728 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.369124 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d47ac-2fab-4e49-bcbf-f9e75568a728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.445533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3e7d47ac-2fab-4e49-bcbf-f9e75568a728 None None] MFA Rules not processed for user `ef8d30e87c624d6180656d998afcebe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:53.446772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3e7d47ac-2fab-4e49-bcbf-f9e75568a728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:53.553871 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 700/1411] 10.222.0.86 () {58 vars in 1015 bytes} [Wed Aug 24 09:56:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:53.569160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e192c0-eb62-46ba-91dc-294d19ffa6be None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.569444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e192c0-eb62-46ba-91dc-294d19ffa6be None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.569715 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e192c0-eb62-46ba-91dc-294d19ffa6be None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.571048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 712/1412] 10.222.0.86 () {54 vars in 918 bytes} [Wed Aug 24 09:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:53.576142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178b1a-74ec-4239-933a-3b4d55037ad0 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:53.576491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178b1a-74ec-4239-933a-3b4d55037ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:53.576770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178b1a-74ec-4239-933a-3b4d55037ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:53.655075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:53.659539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a1178b1a-74ec-4239-933a-3b4d55037ad0 None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:53.661363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a1178b1a-74ec-4239-933a-3b4d55037ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:53.802070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 701/1413] 10.222.0.86 () {58 vars in 1015 bytes} [Wed Aug 24 09:56:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:56.185623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e357c-2ae5-4574-9afd-0bed03d507ef None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:56.186167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e357c-2ae5-4574-9afd-0bed03d507ef None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:56.186167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e357c-2ae5-4574-9afd-0bed03d507ef None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:56.187632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:56.193111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797329c-9e2e-43b7-9fdb-0e58183c4bc0 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:56.193802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797329c-9e2e-43b7-9fdb-0e58183c4bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:56.194017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797329c-9e2e-43b7-9fdb-0e58183c4bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:56.259585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9797329c-9e2e-43b7-9fdb-0e58183c4bc0 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:56.260829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9797329c-9e2e-43b7-9fdb-0e58183c4bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:56.316035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:56.358947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:56.654021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e357c-2ae5-4574-9afd-0bed03d507ef None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:56.655976 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7e5e357c-2ae5-4574-9afd-0bed03d507ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:56.814135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e357c-2ae5-4574-9afd-0bed03d507ef None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:56.887665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:56.928379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a21197-6ba5-45d5-ad09-8905e55fbfe7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:56.928905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a21197-6ba5-45d5-ad09-8905e55fbfe7 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:56.929183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a21197-6ba5-45d5-ad09-8905e55fbfe7 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:56.929460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a21197-6ba5-45d5-ad09-8905e55fbfe7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:56.931050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a4a21197-6ba5-45d5-ad09-8905e55fbfe7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:57.020837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a21197-6ba5-45d5-ad09-8905e55fbfe7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:57.022453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a21197-6ba5-45d5-ad09-8905e55fbfe7 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:57.040644 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 714/1416] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:57.928099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bc95c-77f8-4bdd-940c-bb8ad210ea60 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:57.928674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bc95c-77f8-4bdd-940c-bb8ad210ea60 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:57.928674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bc95c-77f8-4bdd-940c-bb8ad210ea60 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:57.929994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:57.934970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764081a1-0734-4ba7-abe5-d8b32c1d4ce7 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:57.935422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764081a1-0734-4ba7-abe5-d8b32c1d4ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:57.935529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764081a1-0734-4ba7-abe5-d8b32c1d4ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:58.010246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-764081a1-0734-4ba7-abe5-d8b32c1d4ce7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:58.011448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-764081a1-0734-4ba7-abe5-d8b32c1d4ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:58.039412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:58.108574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:58.259379 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bc95c-77f8-4bdd-940c-bb8ad210ea60 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:58.261038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-640bc95c-77f8-4bdd-940c-bb8ad210ea60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:58.407466 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bc95c-77f8-4bdd-940c-bb8ad210ea60 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:58.446350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:58.520584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8314ed0-6c9c-44bf-bbb4-d9db59c49c89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:58.521180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8314ed0-6c9c-44bf-bbb4-d9db59c49c89 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:58.521457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8314ed0-6c9c-44bf-bbb4-d9db59c49c89 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:58.521816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8314ed0-6c9c-44bf-bbb4-d9db59c49c89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:58.523344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e8314ed0-6c9c-44bf-bbb4-d9db59c49c89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:58.610900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8314ed0-6c9c-44bf-bbb4-d9db59c49c89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:58.612793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8314ed0-6c9c-44bf-bbb4-d9db59c49c89 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:58.630736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 704/1419] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:59.509330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b79ee6-f56e-49d6-9f64-06b187aef7d4 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:59.509911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b79ee6-f56e-49d6-9f64-06b187aef7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:59.509911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b79ee6-f56e-49d6-9f64-06b187aef7d4 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:59.511225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:59.516807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae91279-df29-4512-8baa-50a04e2e4bd0 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:59.517191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae91279-df29-4512-8baa-50a04e2e4bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:59.517353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae91279-df29-4512-8baa-50a04e2e4bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:59.572282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:59.594602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7ae91279-df29-4512-8baa-50a04e2e4bd0 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:59.595866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7ae91279-df29-4512-8baa-50a04e2e4bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:59.690478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:56:59.840908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b79ee6-f56e-49d6-9f64-06b187aef7d4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:59.842430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c9b79ee6-f56e-49d6-9f64-06b187aef7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:59.989908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b79ee6-f56e-49d6-9f64-06b187aef7d4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:00.002982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:00.102966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577af3de-412e-4686-a82c-9eb2fac6293f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:00.103483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577af3de-412e-4686-a82c-9eb2fac6293f service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:00.103762 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577af3de-412e-4686-a82c-9eb2fac6293f service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:00.104031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577af3de-412e-4686-a82c-9eb2fac6293f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:00.105683 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-577af3de-412e-4686-a82c-9eb2fac6293f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:00.195245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577af3de-412e-4686-a82c-9eb2fac6293f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:00.196907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577af3de-412e-4686-a82c-9eb2fac6293f service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:00.214383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 717/1422] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:01.073871 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c708636-7fa4-4e13-a061-47e588d96cd9 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.074449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c708636-7fa4-4e13-a061-47e588d96cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.074449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c708636-7fa4-4e13-a061-47e588d96cd9 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.075910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:01.080940 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae2dd4-56e3-4b67-9dc7-7382592a3903 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.081347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae2dd4-56e3-4b67-9dc7-7382592a3903 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.081488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae2dd4-56e3-4b67-9dc7-7382592a3903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.101434 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:01.157517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e0ae2dd4-56e3-4b67-9dc7-7382592a3903 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:01.158730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e0ae2dd4-56e3-4b67-9dc7-7382592a3903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:01.253768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:01.402335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c708636-7fa4-4e13-a061-47e588d96cd9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:01.403788 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7c708636-7fa4-4e13-a061-47e588d96cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:01.511433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:01.548798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c708636-7fa4-4e13-a061-47e588d96cd9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:01.661396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fb166-c296-4909-a6e2-1c5e32db6a4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:01.661917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fb166-c296-4909-a6e2-1c5e32db6a4d service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:01.662199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fb166-c296-4909-a6e2-1c5e32db6a4d service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:01.662475 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fb166-c296-4909-a6e2-1c5e32db6a4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:01.664096 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e45fb166-c296-4909-a6e2-1c5e32db6a4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:01.739283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:01.753407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fb166-c296-4909-a6e2-1c5e32db6a4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:01.755008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fb166-c296-4909-a6e2-1c5e32db6a4d service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:01.773302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 707/1425] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:02.636501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b7539-243a-43c1-aa80-465c00eace69 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.637338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b7539-243a-43c1-aa80-465c00eace69 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.637501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b7539-243a-43c1-aa80-465c00eace69 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.638942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:57:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:02.643917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713eb80-ee1a-4349-bcd6-bbe5b5fd49da None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:02.644464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713eb80-ee1a-4349-bcd6-bbe5b5fd49da None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:02.644559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713eb80-ee1a-4349-bcd6-bbe5b5fd49da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:02.719899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5713eb80-ee1a-4349-bcd6-bbe5b5fd49da None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:02.721133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5713eb80-ee1a-4349-bcd6-bbe5b5fd49da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:02.817589 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:02.969025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b7539-243a-43c1-aa80-465c00eace69 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:02.970560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b68b7539-243a-43c1-aa80-465c00eace69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:03.053608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:03.115382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b7539-243a-43c1-aa80-465c00eace69 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:03.234674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444be8e-609c-4689-b7b6-bdc6d1143480 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:03.235188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444be8e-609c-4689-b7b6-bdc6d1143480 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:03.235470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444be8e-609c-4689-b7b6-bdc6d1143480 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:03.235742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444be8e-609c-4689-b7b6-bdc6d1143480 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:03.237635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1444be8e-609c-4689-b7b6-bdc6d1143480 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:03.286785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:03.328978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444be8e-609c-4689-b7b6-bdc6d1143480 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:03.330595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444be8e-609c-4689-b7b6-bdc6d1143480 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:03.348183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 720/1428] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:04.143926 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5b3a5-a13a-43c1-9009-c3330f9ae69f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:04.144529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5b3a5-a13a-43c1-9009-c3330f9ae69f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:04.144529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5b3a5-a13a-43c1-9009-c3330f9ae69f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:04.145814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:04.150909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400c62c-456a-4bff-ab99-01fd234e0aef None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:04.151366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400c62c-456a-4bff-ab99-01fd234e0aef None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:04.151471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400c62c-456a-4bff-ab99-01fd234e0aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:04.225611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-9400c62c-456a-4bff-ab99-01fd234e0aef None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:04.226813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9400c62c-456a-4bff-ab99-01fd234e0aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:04.323161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:04.475930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5b3a5-a13a-43c1-9009-c3330f9ae69f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:04.477485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b9f5b3a5-a13a-43c1-9009-c3330f9ae69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:04.528584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:04.625877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5b3a5-a13a-43c1-9009-c3330f9ae69f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:04.745445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e513fb-45a5-44a0-a0a5-871f128259b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:04.745968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e513fb-45a5-44a0-a0a5-871f128259b2 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:04.746509 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e513fb-45a5-44a0-a0a5-871f128259b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:04.746786 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e513fb-45a5-44a0-a0a5-871f128259b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:04.748409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-88e513fb-45a5-44a0-a0a5-871f128259b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:04.755311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:04.841260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e513fb-45a5-44a0-a0a5-871f128259b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:04.842943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e513fb-45a5-44a0-a0a5-871f128259b2 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:04.861355 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 710/1431] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:05.667366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d01455-96dc-433a-b711-36fa8f44ef26 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:05.667923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d01455-96dc-433a-b711-36fa8f44ef26 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:05.667923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d01455-96dc-433a-b711-36fa8f44ef26 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:05.669393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:05.675090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c902-9a8a-438e-900d-fb13d513dea1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:05.675537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c902-9a8a-438e-900d-fb13d513dea1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:05.675639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c902-9a8a-438e-900d-fb13d513dea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:05.751338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f1c4c902-9a8a-438e-900d-fb13d513dea1 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:05.752586 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f1c4c902-9a8a-438e-900d-fb13d513dea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:05.820770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:05.850229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:06.002497 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d01455-96dc-433a-b711-36fa8f44ef26 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:06.004080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-05d01455-96dc-433a-b711-36fa8f44ef26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:06.022668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:06.154344 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d01455-96dc-433a-b711-36fa8f44ef26 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:06.244656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:06.271877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161cecb-1fbe-4cb8-bd46-f9d115ba59d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:06.272514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161cecb-1fbe-4cb8-bd46-f9d115ba59d2 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:06.272847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161cecb-1fbe-4cb8-bd46-f9d115ba59d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:06.273073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161cecb-1fbe-4cb8-bd46-f9d115ba59d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:06.274860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7161cecb-1fbe-4cb8-bd46-f9d115ba59d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:06.367616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161cecb-1fbe-4cb8-bd46-f9d115ba59d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:06.369295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161cecb-1fbe-4cb8-bd46-f9d115ba59d2 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:06.387899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 723/1434] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:07.191480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd52a8-b2fb-496c-8eec-00b33563b7a0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:07.192072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd52a8-b2fb-496c-8eec-00b33563b7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:07.192072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd52a8-b2fb-496c-8eec-00b33563b7a0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:07.193517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:07.198379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2bb29-3e1c-41d2-947d-3f517456a02a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:07.198829 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2bb29-3e1c-41d2-947d-3f517456a02a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:07.198935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2bb29-3e1c-41d2-947d-3f517456a02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:07.274036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-99d2bb29-3e1c-41d2-947d-3f517456a02a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:07.275172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-99d2bb29-3e1c-41d2-947d-3f517456a02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:07.317208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:07.373046 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:07.523587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd52a8-b2fb-496c-8eec-00b33563b7a0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:07.525272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6ccd52a8-b2fb-496c-8eec-00b33563b7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:07.669083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd52a8-b2fb-496c-8eec-00b33563b7a0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:07.722797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:07.782793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20c670-7da1-445d-9440-ba4d071291c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:07.783312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c670-7da1-445d-9440-ba4d071291c0 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:07.783588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c670-7da1-445d-9440-ba4d071291c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:07.783862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c670-7da1-445d-9440-ba4d071291c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:07.785550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ac20c670-7da1-445d-9440-ba4d071291c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:07.872932 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20c670-7da1-445d-9440-ba4d071291c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:07.875085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20c670-7da1-445d-9440-ba4d071291c0 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:07.893380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 713/1437] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:08.696825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a122f-f308-4773-a7d5-eaecb6ce2733 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:08.697376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a122f-f308-4773-a7d5-eaecb6ce2733 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:08.697376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a122f-f308-4773-a7d5-eaecb6ce2733 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:08.698768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:08.704957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd593e-bc41-4d33-b677-4d715ca58664 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:08.705413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd593e-bc41-4d33-b677-4d715ca58664 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:08.705703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd593e-bc41-4d33-b677-4d715ca58664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:08.781607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-92bd593e-bc41-4d33-b677-4d715ca58664 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:08.782840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-92bd593e-bc41-4d33-b677-4d715ca58664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:08.785300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:08.882009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:09.030761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a122f-f308-4773-a7d5-eaecb6ce2733 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:09.032219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cc6a122f-f308-4773-a7d5-eaecb6ce2733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:09.181829 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a122f-f308-4773-a7d5-eaecb6ce2733 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:09.209274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:09.293556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52d43d-9d9e-4694-992d-7e09a97be323 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:09.294075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52d43d-9d9e-4694-992d-7e09a97be323 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:09.294353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52d43d-9d9e-4694-992d-7e09a97be323 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:09.294622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52d43d-9d9e-4694-992d-7e09a97be323 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:09.296762 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7e52d43d-9d9e-4694-992d-7e09a97be323 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:09.386732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52d43d-9d9e-4694-992d-7e09a97be323 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:09.388395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52d43d-9d9e-4694-992d-7e09a97be323 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:09.406666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 726/1440] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:11.239675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378189a-8514-47f1-a58c-b4ac7ff3069c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:11.240280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378189a-8514-47f1-a58c-b4ac7ff3069c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:11.240280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378189a-8514-47f1-a58c-b4ac7ff3069c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:11.241641 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:11.247070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348e32b-fe34-4a1e-bac4-ebab5861e0fc None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:11.247552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348e32b-fe34-4a1e-bac4-ebab5861e0fc None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:11.247653 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348e32b-fe34-4a1e-bac4-ebab5861e0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:11.289657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:11.316862 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a348e32b-fe34-4a1e-bac4-ebab5861e0fc None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:11.318416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a348e32b-fe34-4a1e-bac4-ebab5861e0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:11.425657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:11.720073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378189a-8514-47f1-a58c-b4ac7ff3069c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:11.721653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6378189a-8514-47f1-a58c-b4ac7ff3069c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:11.828694 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378189a-8514-47f1-a58c-b4ac7ff3069c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:11.860107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:11.903640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bd9fd-3c31-48a4-874d-7589e257395b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:11.904277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bd9fd-3c31-48a4-874d-7589e257395b service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:11.904584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bd9fd-3c31-48a4-874d-7589e257395b service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:11.904878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bd9fd-3c31-48a4-874d-7589e257395b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:11.906473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-862bd9fd-3c31-48a4-874d-7589e257395b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:12.017641 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bd9fd-3c31-48a4-874d-7589e257395b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:12.019072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bd9fd-3c31-48a4-874d-7589e257395b service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:12.037014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 716/1443] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:14.072858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5566aea-7322-4082-ae00-712fd0eeaabd None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.073403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5566aea-7322-4082-ae00-712fd0eeaabd None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.073403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5566aea-7322-4082-ae00-712fd0eeaabd None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.074747 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:14.080107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5fc39-0437-401e-84cb-b59798839158 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.080538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5fc39-0437-401e-84cb-b59798839158 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.080720 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5fc39-0437-401e-84cb-b59798839158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.149132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f3c5fc39-0437-401e-84cb-b59798839158 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:14.150332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f3c5fc39-0437-401e-84cb-b59798839158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:14.163490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:14.248240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:14.531645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5566aea-7322-4082-ae00-712fd0eeaabd None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:14.533256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a5566aea-7322-4082-ae00-712fd0eeaabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:14.646663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5566aea-7322-4082-ae00-712fd0eeaabd None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:14.662507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:14.725009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588294f-8ac7-46e0-80a4-d4e5878b05a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:14.725614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588294f-8ac7-46e0-80a4-d4e5878b05a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:14.725896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588294f-8ac7-46e0-80a4-d4e5878b05a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:14.726174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588294f-8ac7-46e0-80a4-d4e5878b05a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:14.727769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7588294f-8ac7-46e0-80a4-d4e5878b05a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:14.837708 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588294f-8ac7-46e0-80a4-d4e5878b05a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:14.839079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588294f-8ac7-46e0-80a4-d4e5878b05a3 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:14.857079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 729/1446] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:17.063935 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf447d7f-1401-4234-b562-4e07891a4467 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:17.064495 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf447d7f-1401-4234-b562-4e07891a4467 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:17.064495 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf447d7f-1401-4234-b562-4e07891a4467 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:17.065794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:17.071302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c402754-64c2-4800-a0d0-2b832a424afa None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:17.071804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c402754-64c2-4800-a0d0-2b832a424afa None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:17.071908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c402754-64c2-4800-a0d0-2b832a424afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:17.120336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:17.139318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2c402754-64c2-4800-a0d0-2b832a424afa None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:17.140557 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2c402754-64c2-4800-a0d0-2b832a424afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:17.234044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:17.517119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf447d7f-1401-4234-b562-4e07891a4467 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:17.518661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bf447d7f-1401-4234-b562-4e07891a4467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:17.623207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf447d7f-1401-4234-b562-4e07891a4467 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:17.662524 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:17.697026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd2bd5-b13c-4a96-96ef-db4b67fe6e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:17.697628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd2bd5-b13c-4a96-96ef-db4b67fe6e6f service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:17.697904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd2bd5-b13c-4a96-96ef-db4b67fe6e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:17.698172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd2bd5-b13c-4a96-96ef-db4b67fe6e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:17.699748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c8dd2bd5-b13c-4a96-96ef-db4b67fe6e6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:17.809547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd2bd5-b13c-4a96-96ef-db4b67fe6e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:17.811034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd2bd5-b13c-4a96-96ef-db4b67fe6e6f service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:17.828663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 719/1449] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:19.847231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00d1c3-1ab6-4353-854d-a4599b5db45d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:19.847825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00d1c3-1ab6-4353-854d-a4599b5db45d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:19.847825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00d1c3-1ab6-4353-854d-a4599b5db45d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:19.849185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:19.854402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c02c3-0c48-4b9f-aeb6-5a55d0e8bb15 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:19.854833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c02c3-0c48-4b9f-aeb6-5a55d0e8bb15 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:19.854963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c02c3-0c48-4b9f-aeb6-5a55d0e8bb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:19.924102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-690c02c3-0c48-4b9f-aeb6-5a55d0e8bb15 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:19.925333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-690c02c3-0c48-4b9f-aeb6-5a55d0e8bb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:19.952019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:20.030268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:20.317741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00d1c3-1ab6-4353-854d-a4599b5db45d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:20.319305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7e00d1c3-1ab6-4353-854d-a4599b5db45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:20.422420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00d1c3-1ab6-4353-854d-a4599b5db45d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:20.442488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:20.496574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f2a89-fa59-4672-8d8e-eb691de82a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:20.497167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f2a89-fa59-4672-8d8e-eb691de82a13 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:20.497443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f2a89-fa59-4672-8d8e-eb691de82a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:20.497709 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f2a89-fa59-4672-8d8e-eb691de82a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:20.499842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1a8f2a89-fa59-4672-8d8e-eb691de82a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:20.606892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f2a89-fa59-4672-8d8e-eb691de82a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:20.608901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f2a89-fa59-4672-8d8e-eb691de82a13 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:20.627915 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 732/1452] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:22.579466 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547391-69f1-4165-9657-6a4b1d1b0846 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:22.580075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547391-69f1-4165-9657-6a4b1d1b0846 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:22.580075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547391-69f1-4165-9657-6a4b1d1b0846 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:22.581438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:22.586904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ef34e-a6a6-43e0-bb88-cfd73ea54a65 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:22.587329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ef34e-a6a6-43e0-bb88-cfd73ea54a65 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:22.587449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ef34e-a6a6-43e0-bb88-cfd73ea54a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:22.649889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:22.652821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e26ef34e-a6a6-43e0-bb88-cfd73ea54a65 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:22.655371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e26ef34e-a6a6-43e0-bb88-cfd73ea54a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:22.752721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:57:23.034670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d547391-69f1-4165-9657-6a4b1d1b0846 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:23.036257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2d547391-69f1-4165-9657-6a4b1d1b0846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:23.139580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d547391-69f1-4165-9657-6a4b1d1b0846 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:23.188298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:23.213481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9e58e-6979-4c75-bc2b-65ebb76a9a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:23.214077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9e58e-6979-4c75-bc2b-65ebb76a9a42 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:23.214351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9e58e-6979-4c75-bc2b-65ebb76a9a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:23.214620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9e58e-6979-4c75-bc2b-65ebb76a9a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:23.216225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-81c9e58e-6979-4c75-bc2b-65ebb76a9a42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:23.325047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9e58e-6979-4c75-bc2b-65ebb76a9a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:23.326487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9e58e-6979-4c75-bc2b-65ebb76a9a42 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:23.344683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 722/1455] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:25.319926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed3cf-2d24-43d1-b999-9d033baa3ba5 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:25.320537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed3cf-2d24-43d1-b999-9d033baa3ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:25.320537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed3cf-2d24-43d1-b999-9d033baa3ba5 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:25.321844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:25.327878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a111fd-f874-46b8-8afd-7affdf4733b8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:25.328333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a111fd-f874-46b8-8afd-7affdf4733b8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:25.328448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a111fd-f874-46b8-8afd-7affdf4733b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:25.394980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-60a111fd-f874-46b8-8afd-7affdf4733b8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:25.396228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-60a111fd-f874-46b8-8afd-7affdf4733b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:25.425900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:25.494448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:57:25.773457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bed3cf-2d24-43d1-b999-9d033baa3ba5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:25.775055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d0bed3cf-2d24-43d1-b999-9d033baa3ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:25.878718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bed3cf-2d24-43d1-b999-9d033baa3ba5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:25.907012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:25.952895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec93e5-a0df-465f-abab-111a9f7ed593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:25.953453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec93e5-a0df-465f-abab-111a9f7ed593 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:25.953734 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec93e5-a0df-465f-abab-111a9f7ed593 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:25.954007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec93e5-a0df-465f-abab-111a9f7ed593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:25.955601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f9ec93e5-a0df-465f-abab-111a9f7ed593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:26.064365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec93e5-a0df-465f-abab-111a9f7ed593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:26.065815 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec93e5-a0df-465f-abab-111a9f7ed593 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:26.085474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 735/1458] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:28.022210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a4a9d-80b2-4c0b-8b35-bce588a14cef None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:28.022750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a4a9d-80b2-4c0b-8b35-bce588a14cef None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:28.022750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a4a9d-80b2-4c0b-8b35-bce588a14cef None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:28.024144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:28.029392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922bbdf-a8be-456f-9f78-b6ff30cba9a3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:28.029790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922bbdf-a8be-456f-9f78-b6ff30cba9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:28.029937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922bbdf-a8be-456f-9f78-b6ff30cba9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:28.095456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0922bbdf-a8be-456f-9f78-b6ff30cba9a3 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:28.096685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0922bbdf-a8be-456f-9f78-b6ff30cba9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:28.104920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:28.192629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:28.478187 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a4a9d-80b2-4c0b-8b35-bce588a14cef None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:28.479760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dd2a4a9d-80b2-4c0b-8b35-bce588a14cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:28.583951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a4a9d-80b2-4c0b-8b35-bce588a14cef None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:28.639464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:28.658673 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80a943-d51f-4eed-9f51-638208adb4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:28.659281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80a943-d51f-4eed-9f51-638208adb4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:28.659557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80a943-d51f-4eed-9f51-638208adb4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:28.659826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80a943-d51f-4eed-9f51-638208adb4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:28.661450 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1c80a943-d51f-4eed-9f51-638208adb4a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:28.770210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80a943-d51f-4eed-9f51-638208adb4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:28.771659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80a943-d51f-4eed-9f51-638208adb4a0 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:28.790240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 725/1461] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:30.734556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b0758-122e-4c6d-bb44-7fcf7ad7fd11 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:30.735145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b0758-122e-4c6d-bb44-7fcf7ad7fd11 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:30.735145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b0758-122e-4c6d-bb44-7fcf7ad7fd11 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:30.736739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:30.742062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cb5e9-d80b-4fda-af86-674cf8d781ef None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:30.742552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cb5e9-d80b-4fda-af86-674cf8d781ef None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:30.742653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cb5e9-d80b-4fda-af86-674cf8d781ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:30.809454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0c5cb5e9-d80b-4fda-af86-674cf8d781ef None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:30.811267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0c5cb5e9-d80b-4fda-af86-674cf8d781ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:30.849262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:30.908161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:57:31.189079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b0758-122e-4c6d-bb44-7fcf7ad7fd11 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:31.190609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-305b0758-122e-4c6d-bb44-7fcf7ad7fd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:31.293798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b0758-122e-4c6d-bb44-7fcf7ad7fd11 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:31.330648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:31.368239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db47ed-17bb-4d92-ab2c-3bb65b1ae703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:31.368833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db47ed-17bb-4d92-ab2c-3bb65b1ae703 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:31.369110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db47ed-17bb-4d92-ab2c-3bb65b1ae703 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:31.369376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db47ed-17bb-4d92-ab2c-3bb65b1ae703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:31.370949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-01db47ed-17bb-4d92-ab2c-3bb65b1ae703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:31.478131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db47ed-17bb-4d92-ab2c-3bb65b1ae703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:31.479487 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db47ed-17bb-4d92-ab2c-3bb65b1ae703 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:31.497445 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 738/1464] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:33.513945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb97c3e-268f-4e67-821b-78023712fa24 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.514518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb97c3e-268f-4e67-821b-78023712fa24 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.514518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb97c3e-268f-4e67-821b-78023712fa24 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.515859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:33.521336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4b8f2-d783-4aee-a834-ac353486d09c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.521741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4b8f2-d783-4aee-a834-ac353486d09c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.521892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4b8f2-d783-4aee-a834-ac353486d09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.587891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-29f4b8f2-d783-4aee-a834-ac353486d09c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:33.589130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-29f4b8f2-d783-4aee-a834-ac353486d09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:33.608176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:33.686026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:33.972496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb97c3e-268f-4e67-821b-78023712fa24 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:33.974087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9fb97c3e-268f-4e67-821b-78023712fa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:34.083938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb97c3e-268f-4e67-821b-78023712fa24 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:34.149389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:34.156373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633e37b-c67c-4cfb-82d7-e84360660c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:34.156971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633e37b-c67c-4cfb-82d7-e84360660c76 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:34.157249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633e37b-c67c-4cfb-82d7-e84360660c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:34.157519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633e37b-c67c-4cfb-82d7-e84360660c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:34.159126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9633e37b-c67c-4cfb-82d7-e84360660c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:34.269306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633e37b-c67c-4cfb-82d7-e84360660c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:34.270738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633e37b-c67c-4cfb-82d7-e84360660c76 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:34.288491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 728/1467] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:36.224703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0561ca-e9a8-4064-b18d-2fe20e6c804b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.225289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0561ca-e9a8-4064-b18d-2fe20e6c804b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.225289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0561ca-e9a8-4064-b18d-2fe20e6c804b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.226641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:36.232469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb97629-ba87-4b33-b533-4fa38e0a99a5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.232911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb97629-ba87-4b33-b533-4fa38e0a99a5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.233027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb97629-ba87-4b33-b533-4fa38e0a99a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.302151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-efb97629-ba87-4b33-b533-4fa38e0a99a5 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:36.303356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-efb97629-ba87-4b33-b533-4fa38e0a99a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:36.351532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:36.400648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:36.683483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0561ca-e9a8-4064-b18d-2fe20e6c804b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.685146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1d0561ca-e9a8-4064-b18d-2fe20e6c804b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:36.791832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0561ca-e9a8-4064-b18d-2fe20e6c804b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.840069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:36.867208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f3b2b-3f39-4f7a-9499-ce8508ac0516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.867805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f3b2b-3f39-4f7a-9499-ce8508ac0516 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.868105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f3b2b-3f39-4f7a-9499-ce8508ac0516 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.868375 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f3b2b-3f39-4f7a-9499-ce8508ac0516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.869947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1e4f3b2b-3f39-4f7a-9499-ce8508ac0516 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:36.982027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f3b2b-3f39-4f7a-9499-ce8508ac0516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.983488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f3b2b-3f39-4f7a-9499-ce8508ac0516 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.005720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 741/1470] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:46.398912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=22 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:47.000310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=66 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:56.535340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d6cb-0aa5-4d02-a9f3-2df796d813f8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.535911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d6cb-0aa5-4d02-a9f3-2df796d813f8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.535911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593d6cb-0aa5-4d02-a9f3-2df796d813f8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.537380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 730/1471] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:56.543027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4071b67-83ca-467d-a20c-6015683ba0bd None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.543462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4071b67-83ca-467d-a20c-6015683ba0bd None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.543562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4071b67-83ca-467d-a20c-6015683ba0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.610738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c4071b67-83ca-467d-a20c-6015683ba0bd None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:56.611939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c4071b67-83ca-467d-a20c-6015683ba0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:56.705622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 742/1472] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:56.997904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6152d-7f48-49b7-978d-b110069b4b60 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.998190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6152d-7f48-49b7-978d-b110069b4b60 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.998464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6152d-7f48-49b7-978d-b110069b4b60 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.999789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 731/1473] 10.222.0.86 () {54 vars in 967 bytes} [Wed Aug 24 09:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:57.004826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cfa9c-8147-4f03-bf83-3b020cf90f65 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.005146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cfa9c-8147-4f03-bf83-3b020cf90f65 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.005395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cfa9c-8147-4f03-bf83-3b020cf90f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.080642 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0a5cfa9c-8147-4f03-bf83-3b020cf90f65 None None] MFA Rules not processed for user `794dbf38682e4b4f939854eecbacaf07`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:57.081834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0a5cfa9c-8147-4f03-bf83-3b020cf90f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:57.102117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:57.158227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 743/1474] 10.222.0.86 () {58 vars in 1064 bytes} [Wed Aug 24 09:57:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:57.165697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6152d-7f48-49b7-978d-b110069b4b60 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:57.167211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2cc6152d-7f48-49b7-978d-b110069b4b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:57.270989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6152d-7f48-49b7-978d-b110069b4b60 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:57.346642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427da8d-1e78-40ae-b6ab-a269f39bfabe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '794dbf38682e4b4f939854eecbacaf07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:57.347251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427da8d-1e78-40ae-b6ab-a269f39bfabe service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.347533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427da8d-1e78-40ae-b6ab-a269f39bfabe service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.347828 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427da8d-1e78-40ae-b6ab-a269f39bfabe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.349464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3427da8d-1e78-40ae-b6ab-a269f39bfabe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:57.404759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:57.459935 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427da8d-1e78-40ae-b6ab-a269f39bfabe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:57.461399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427da8d-1e78-40ae-b6ab-a269f39bfabe service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:57.479359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 732/1475] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:59.484053 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b4b13-0de8-4ae8-9ad4-8f7e7b667d9e None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:59.484858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b4b13-0de8-4ae8-9ad4-8f7e7b667d9e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:59.484956 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b4b13-0de8-4ae8-9ad4-8f7e7b667d9e None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:59.486267 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 744/1476] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:59.491633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d79c8a-5f34-45d1-8a7a-89987a627ed8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:59.492102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d79c8a-5f34-45d1-8a7a-89987a627ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:59.492203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d79c8a-5f34-45d1-8a7a-89987a627ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:59.557035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c8d79c8a-5f34-45d1-8a7a-89987a627ed8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:59.558883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c8d79c8a-5f34-45d1-8a7a-89987a627ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:59.653117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 733/1477] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:59.923819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b4b13-0de8-4ae8-9ad4-8f7e7b667d9e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:59.925389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e27b4b13-0de8-4ae8-9ad4-8f7e7b667d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:00.051730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b4b13-0de8-4ae8-9ad4-8f7e7b667d9e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:00.100761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:00.146429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3cb50-0ab6-4f7a-a122-254044b07e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:00.147034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3cb50-0ab6-4f7a-a122-254044b07e65 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:00.147313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3cb50-0ab6-4f7a-a122-254044b07e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:00.147585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3cb50-0ab6-4f7a-a122-254044b07e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:00.148257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3cb50-0ab6-4f7a-a122-254044b07e65 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:00.150070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-f6f3cb50-0ab6-4f7a-a122-254044b07e65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:58:00.150070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Aug 24 09:58:00.150070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: Aug 24 09:58:00.151861 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3cb50-0ab6-4f7a-a122-254044b07e65 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:00.170763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 745/1478] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Aug 24 09:57:59 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:58:02.104895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8188db5-38ee-40ea-ae5a-635cd0354534 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.105476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8188db5-38ee-40ea-ae5a-635cd0354534 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.105476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8188db5-38ee-40ea-ae5a-635cd0354534 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.106784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 734/1479] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:02.112209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719437f9-726a-4872-aaf7-29ec8555120b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.112682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719437f9-726a-4872-aaf7-29ec8555120b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.112793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719437f9-726a-4872-aaf7-29ec8555120b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.177045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-719437f9-726a-4872-aaf7-29ec8555120b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:02.178242 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-719437f9-726a-4872-aaf7-29ec8555120b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:02.272464 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 746/1480] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:02.568302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8188db5-38ee-40ea-ae5a-635cd0354534 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:02.569923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c8188db5-38ee-40ea-ae5a-635cd0354534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:02.579996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:02.717960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8188db5-38ee-40ea-ae5a-635cd0354534 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:02.799804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:02.832560 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5107dac-01ad-4654-8322-020218a4cf29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:02.833082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5107dac-01ad-4654-8322-020218a4cf29 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:02.833363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5107dac-01ad-4654-8322-020218a4cf29 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:02.833635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5107dac-01ad-4654-8322-020218a4cf29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:02.835226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a5107dac-01ad-4654-8322-020218a4cf29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:02.924255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5107dac-01ad-4654-8322-020218a4cf29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:02.925875 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5107dac-01ad-4654-8322-020218a4cf29 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:02.943564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 735/1481] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:05.058342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7e47-813a-4894-adbd-8601f3cfd72d None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.058990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7e47-813a-4894-adbd-8601f3cfd72d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.058990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7e47-813a-4894-adbd-8601f3cfd72d None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.060430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 747/1482] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:05.065493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b58768-9cff-4e48-bc4c-c2ab24818937 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.065964 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b58768-9cff-4e48-bc4c-c2ab24818937 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.066151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b58768-9cff-4e48-bc4c-c2ab24818937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.131184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-88b58768-9cff-4e48-bc4c-c2ab24818937 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:05.132402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-88b58768-9cff-4e48-bc4c-c2ab24818937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:05.180371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:05.230759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 736/1483] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:05.517114 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be7e47-813a-4894-adbd-8601f3cfd72d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:05.518517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c5be7e47-813a-4894-adbd-8601f3cfd72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:05.542491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:05.623247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be7e47-813a-4894-adbd-8601f3cfd72d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:05.695984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9e3fd-3a95-4dd0-96c2-3dd79dcb83b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:05.696616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9e3fd-3a95-4dd0-96c2-3dd79dcb83b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:05.696891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9e3fd-3a95-4dd0-96c2-3dd79dcb83b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:05.697159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9e3fd-3a95-4dd0-96c2-3dd79dcb83b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:05.699327 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-acd9e3fd-3a95-4dd0-96c2-3dd79dcb83b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:05.772479 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:05.807747 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9e3fd-3a95-4dd0-96c2-3dd79dcb83b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:05.809233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9e3fd-3a95-4dd0-96c2-3dd79dcb83b0 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:05.827654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 748/1484] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:07.799822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e8333-6879-4df7-9346-da98edc7c442 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:07.800360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e8333-6879-4df7-9346-da98edc7c442 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:07.800464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e8333-6879-4df7-9346-da98edc7c442 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:07.801790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 737/1485] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:07.807868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cbba0-b6b5-461f-ac52-7866fafce4d3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:07.808394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cbba0-b6b5-461f-ac52-7866fafce4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:07.808500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cbba0-b6b5-461f-ac52-7866fafce4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:07.874095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2b8cbba0-b6b5-461f-ac52-7866fafce4d3 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:07.875307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2b8cbba0-b6b5-461f-ac52-7866fafce4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:07.970443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 749/1486] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:08.256536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e8333-6879-4df7-9346-da98edc7c442 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:08.258084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6a0e8333-6879-4df7-9346-da98edc7c442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:08.371916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e8333-6879-4df7-9346-da98edc7c442 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:08.442912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:08.444806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767a1ae-b532-4bb5-aadb-cfa5c2ea8ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:08.445447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767a1ae-b532-4bb5-aadb-cfa5c2ea8ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.445917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767a1ae-b532-4bb5-aadb-cfa5c2ea8ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.446358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767a1ae-b532-4bb5-aadb-cfa5c2ea8ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.448667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7767a1ae-b532-4bb5-aadb-cfa5c2ea8ce4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:08.557789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767a1ae-b532-4bb5-aadb-cfa5c2ea8ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:08.559167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767a1ae-b532-4bb5-aadb-cfa5c2ea8ce4 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:08.577296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 738/1487] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:10.588352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd11e-c7dc-46f1-82f3-d70ace9d4521 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:10.588925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd11e-c7dc-46f1-82f3-d70ace9d4521 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:10.588925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cd11e-c7dc-46f1-82f3-d70ace9d4521 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:10.590329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 750/1488] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:10.595862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f9b70-fdd4-4072-b2ce-6c3c5d93489f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:10.596331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f9b70-fdd4-4072-b2ce-6c3c5d93489f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:10.596494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f9b70-fdd4-4072-b2ce-6c3c5d93489f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:10.661358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5f4f9b70-fdd4-4072-b2ce-6c3c5d93489f None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:10.662570 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5f4f9b70-fdd4-4072-b2ce-6c3c5d93489f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:10.725880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:10.771691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 739/1489] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:11.072546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cd11e-c7dc-46f1-82f3-d70ace9d4521 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:11.074282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5a9cd11e-c7dc-46f1-82f3-d70ace9d4521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:11.111110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:11.177925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cd11e-c7dc-46f1-82f3-d70ace9d4521 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:11.250997 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670bdaf-9dc4-4abb-8907-be062077398b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:11.251597 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670bdaf-9dc4-4abb-8907-be062077398b service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:11.251871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670bdaf-9dc4-4abb-8907-be062077398b service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:11.252169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670bdaf-9dc4-4abb-8907-be062077398b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:11.253749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1670bdaf-9dc4-4abb-8907-be062077398b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:11.332477 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:11.362738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670bdaf-9dc4-4abb-8907-be062077398b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:11.364453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670bdaf-9dc4-4abb-8907-be062077398b service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:11.382313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 751/1490] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:13.469370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126d13b-7d90-4bb6-8381-603fc4622628 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.470108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126d13b-7d90-4bb6-8381-603fc4622628 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.470108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126d13b-7d90-4bb6-8381-603fc4622628 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.471301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 740/1491] 10.222.0.86 () {54 vars in 927 bytes} [Wed Aug 24 09:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:13.476640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6c99d-9633-4f43-9cd9-eec0129ae4bb None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:13.477057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6c99d-9633-4f43-9cd9-eec0129ae4bb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:13.477199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6c99d-9633-4f43-9cd9-eec0129ae4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:13.542660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-74d6c99d-9633-4f43-9cd9-eec0129ae4bb None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:13.543872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-74d6c99d-9633-4f43-9cd9-eec0129ae4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:13.640461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 752/1492] 10.222.0.86 () {58 vars in 1024 bytes} [Wed Aug 24 09:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:13.927358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126d13b-7d90-4bb6-8381-603fc4622628 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:13.928907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7126d13b-7d90-4bb6-8381-603fc4622628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:14.031124 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126d13b-7d90-4bb6-8381-603fc4622628 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:14.104385 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8c0b2-9125-4ed2-919b-602329e90941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:14.104992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8c0b2-9125-4ed2-919b-602329e90941 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.105275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8c0b2-9125-4ed2-919b-602329e90941 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.105544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8c0b2-9125-4ed2-919b-602329e90941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.107145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a1c8c0b2-9125-4ed2-919b-602329e90941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:14.116644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:14.217283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8c0b2-9125-4ed2-919b-602329e90941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:14.218711 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8c0b2-9125-4ed2-919b-602329e90941 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:14.236951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 741/1493] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:23.638649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=82 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:24.234868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=50 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:24.876634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43924f6-a75f-44cd-98dd-ea798571ad31 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:24.877228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43924f6-a75f-44cd-98dd-ea798571ad31 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:24.877228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43924f6-a75f-44cd-98dd-ea798571ad31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:24.954342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e43924f6-a75f-44cd-98dd-ea798571ad31 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:24.955540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e43924f6-a75f-44cd-98dd-ea798571ad31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:25.053895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 753/1494] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:25.062513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e89eb-9c1d-424d-88d8-20de7bfdfadf None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:25.063006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e89eb-9c1d-424d-88d8-20de7bfdfadf None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:25.063109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e89eb-9c1d-424d-88d8-20de7bfdfadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:25.139112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ca8e89eb-9c1d-424d-88d8-20de7bfdfadf None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:25.140341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ca8e89eb-9c1d-424d-88d8-20de7bfdfadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:25.235732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 742/1495] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:25.240432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43924f6-a75f-44cd-98dd-ea798571ad31 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:25.242171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e43924f6-a75f-44cd-98dd-ea798571ad31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:25.253520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43924f6-a75f-44cd-98dd-ea798571ad31 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:25.314144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:25.350822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806beca1-1f5a-4ab9-bb59-e4e0599df039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:25.351424 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806beca1-1f5a-4ab9-bb59-e4e0599df039 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:25.351703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806beca1-1f5a-4ab9-bb59-e4e0599df039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:25.351970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806beca1-1f5a-4ab9-bb59-e4e0599df039 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:25.352643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806beca1-1f5a-4ab9-bb59-e4e0599df039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:25.352943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806beca1-1f5a-4ab9-bb59-e4e0599df039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:25.353953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-806beca1-1f5a-4ab9-bb59-e4e0599df039 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:25.353953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Aug 24 09:59:25.353953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: Aug 24 09:59:25.354867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806beca1-1f5a-4ab9-bb59-e4e0599df039 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:25.366901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 754/1496] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 09:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:25.375162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e89eb-9c1d-424d-88d8-20de7bfdfadf None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:25.384953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e89eb-9c1d-424d-88d8-20de7bfdfadf None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:25.447409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:25.482596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e962b-f3eb-4be9-836e-8e5c9fa9eeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:25.483464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e962b-f3eb-4be9-836e-8e5c9fa9eeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:25.483746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e962b-f3eb-4be9-836e-8e5c9fa9eeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:25.484020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e962b-f3eb-4be9-836e-8e5c9fa9eeb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:25.484684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e962b-f3eb-4be9-836e-8e5c9fa9eeb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:25.486473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-255e962b-f3eb-4be9-836e-8e5c9fa9eeb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:25.486473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 24 09:59:25.486473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: Aug 24 09:59:25.487703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e962b-f3eb-4be9-836e-8e5c9fa9eeb2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:25.543922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 743/1497] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:25 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:25.548122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806beca1-1f5a-4ab9-bb59-e4e0599df039 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:25.549645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-806beca1-1f5a-4ab9-bb59-e4e0599df039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:25.700882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806beca1-1f5a-4ab9-bb59-e4e0599df039 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:25.795977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f18e97-6bc5-44a4-a2ab-5d0d549fe6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:25.796601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f18e97-6bc5-44a4-a2ab-5d0d549fe6c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:25.796875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f18e97-6bc5-44a4-a2ab-5d0d549fe6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:25.797143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f18e97-6bc5-44a4-a2ab-5d0d549fe6c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:25.797754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f18e97-6bc5-44a4-a2ab-5d0d549fe6c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:25.798757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-15f18e97-6bc5-44a4-a2ab-5d0d549fe6c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:25.798757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Aug 24 09:59:25.798757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: Aug 24 09:59:25.799949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f18e97-6bc5-44a4-a2ab-5d0d549fe6c6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:25.841228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 755/1498] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:25 2022] POST /identity/v3/users => generated 290 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:25.845559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e962b-f3eb-4be9-836e-8e5c9fa9eeb2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:25.847138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-255e962b-f3eb-4be9-836e-8e5c9fa9eeb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:25.910008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e962b-f3eb-4be9-836e-8e5c9fa9eeb2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:25.996730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:26.009361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4fcd-0010-4157-a3bd-c7a08e7bf242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:26.009967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4fcd-0010-4157-a3bd-c7a08e7bf242 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:26.010248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4fcd-0010-4157-a3bd-c7a08e7bf242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:26.010524 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4fcd-0010-4157-a3bd-c7a08e7bf242 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:26.011168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e4fcd-0010-4157-a3bd-c7a08e7bf242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:26.012213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-a56e4fcd-0010-4157-a3bd-c7a08e7bf242 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:26.012213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 24 09:59:26.012213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: Aug 24 09:59:26.013105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e4fcd-0010-4157-a3bd-c7a08e7bf242 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:26.025933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 744/1499] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:26.030302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f18e97-6bc5-44a4-a2ab-5d0d549fe6c6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:26.031827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-15f18e97-6bc5-44a4-a2ab-5d0d549fe6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:26.043651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f18e97-6bc5-44a4-a2ab-5d0d549fe6c6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:26.068917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:26.139923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b2803-1433-4b08-8f83-7d9c2abd2131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:26.140553 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b2803-1433-4b08-8f83-7d9c2abd2131 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:26.140830 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b2803-1433-4b08-8f83-7d9c2abd2131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:26.141102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b2803-1433-4b08-8f83-7d9c2abd2131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:26.141738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b2803-1433-4b08-8f83-7d9c2abd2131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:26.142789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-653b2803-1433-4b08-8f83-7d9c2abd2131 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:26.142789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Aug 24 09:59:26.142789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: Aug 24 09:59:26.143674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b2803-1433-4b08-8f83-7d9c2abd2131 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:26.156407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 756/1500] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:26.160962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4fcd-0010-4157-a3bd-c7a08e7bf242 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:26.162455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a56e4fcd-0010-4157-a3bd-c7a08e7bf242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:26.197648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4fcd-0010-4157-a3bd-c7a08e7bf242 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:26.411639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb728b5-eead-4ff8-b789-27ddb69985cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:26.412232 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb728b5-eead-4ff8-b789-27ddb69985cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:26.412512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb728b5-eead-4ff8-b789-27ddb69985cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:26.412783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb728b5-eead-4ff8-b789-27ddb69985cc admin admin] PATH_INFO: `/v3/projects/3a4868eeca2540fa857ab5e60c705653/users/76c2f9573fa84cc197559ee0822da250/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:26.434147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb728b5-eead-4ff8-b789-27ddb69985cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4868eeca2540fa857ab5e60c705653, user_id=76c2f9573fa84cc197559ee0822da250, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:26.435167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-edb728b5-eead-4ff8-b789-27ddb69985cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:26.435167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 24 09:59:26.435167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: Aug 24 09:59:26.437164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb728b5-eead-4ff8-b789-27ddb69985cc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:26.463014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 745/1501] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:26 2022] PUT /identity/v3/projects/3a4868eeca2540fa857ab5e60c705653/users/76c2f9573fa84cc197559ee0822da250/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:26.475384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f67afc-7b7b-440f-a156-f46b20aa167d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:26.475672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f67afc-7b7b-440f-a156-f46b20aa167d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:26.475948 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f67afc-7b7b-440f-a156-f46b20aa167d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:26.532553 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f5f67afc-7b7b-440f-a156-f46b20aa167d None None] MFA Rules not processed for user `76c2f9573fa84cc197559ee0822da250`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:26.533734 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f5f67afc-7b7b-440f-a156-f46b20aa167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:26.608515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 757/1502] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:26.622182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f40b7-19ce-4be1-83ab-b6a4c4c17671 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:26.622503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f40b7-19ce-4be1-83ab-b6a4c4c17671 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:26.622808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f40b7-19ce-4be1-83ab-b6a4c4c17671 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:26.624220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 746/1503] 10.222.0.86 () {54 vars in 964 bytes} [Wed Aug 24 09:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:26.628855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a3df2-2023-47ce-ba41-87b82252412f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:26.629140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a3df2-2023-47ce-ba41-87b82252412f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:26.629410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a3df2-2023-47ce-ba41-87b82252412f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:26.660586 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:26.705488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7a2a3df2-2023-47ce-ba41-87b82252412f None None] MFA Rules not processed for user `bd7b580705f247699aeb58772e668730`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:26.706673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7a2a3df2-2023-47ce-ba41-87b82252412f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:26.779972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 758/1504] 10.222.0.86 () {58 vars in 1061 bytes} [Wed Aug 24 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:26.788166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f40b7-19ce-4be1-83ab-b6a4c4c17671 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:26.789685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-355f40b7-19ce-4be1-83ab-b6a4c4c17671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:26.839332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:26.894936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f40b7-19ce-4be1-83ab-b6a4c4c17671 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:26.968543 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf1ae4-9e08-4269-aab2-904a28a51289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:26.969061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf1ae4-9e08-4269-aab2-904a28a51289 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:26.969340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf1ae4-9e08-4269-aab2-904a28a51289 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:26.969612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf1ae4-9e08-4269-aab2-904a28a51289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:26.971216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f3cf1ae4-9e08-4269-aab2-904a28a51289 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:27.069580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf1ae4-9e08-4269-aab2-904a28a51289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:27.071014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf1ae4-9e08-4269-aab2-904a28a51289 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:27.074519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:27.089429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 747/1505] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:27.134590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2e425-c855-47da-9268-bb233c4dac68 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:27.135007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2e425-c855-47da-9268-bb233c4dac68 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:27.135146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2e425-c855-47da-9268-bb233c4dac68 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:27.136451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 759/1506] 10.222.0.86 () {52 vars in 830 bytes} [Wed Aug 24 09:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:27.140010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf1ae4-9e08-4269-aab2-904a28a51289 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:27.141230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f3cf1ae4-9e08-4269-aab2-904a28a51289 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:27.238619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf1ae4-9e08-4269-aab2-904a28a51289 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:27.331518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:27.332164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:27.332446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:27.332718 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] PATH_INFO: `/v3/users/76c2f9573fa84cc197559ee0822da250` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:27.345223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76c2f9573fa84cc197559ee0822da250)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:27.346253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:27.346253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 24 09:59:27.346253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: Aug 24 09:59:27.347472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:27.357594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:27.414737 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76c2f9573fa84cc197559ee0822da250'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:27.415178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76c2f9573fa84cc197559ee0822da250'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:27.415715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76c2f9573fa84cc197559ee0822da250'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:27.425692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76c2f9573fa84cc197559ee0822da250'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:27.439827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76c2f9573fa84cc197559ee0822da250'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:27.467527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 748/1507] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 09:59:27 2022] DELETE /identity/v3/users/76c2f9573fa84cc197559ee0822da250 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:27.471872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf631dc-f994-4f0b-bee3-c5f94ddc973c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:27.472178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf631dc-f994-4f0b-bee3-c5f94ddc973c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:27.472453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf631dc-f994-4f0b-bee3-c5f94ddc973c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:27.529291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-abf631dc-f994-4f0b-bee3-c5f94ddc973c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:27.530406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-abf631dc-f994-4f0b-bee3-c5f94ddc973c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:27.603750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:27.616660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 760/1508] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 09:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:27.625961 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:27.627209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:27.727852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5e28-a518-456b-abcf-00ea0aa2d624 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:27.784824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:27.806453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4f085-a3a0-4cdb-a687-a0393b2c50bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:27.807056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f085-a3a0-4cdb-a687-a0393b2c50bd service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:27.807331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f085-a3a0-4cdb-a687-a0393b2c50bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:27.807597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f085-a3a0-4cdb-a687-a0393b2c50bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:27.809213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1fe4f085-a3a0-4cdb-a687-a0393b2c50bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:27.914552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4f085-a3a0-4cdb-a687-a0393b2c50bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:27.915982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4f085-a3a0-4cdb-a687-a0393b2c50bd service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:27.933829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 749/1509] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:28.268513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf631dc-f994-4f0b-bee3-c5f94ddc973c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:28.269822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-abf631dc-f994-4f0b-bee3-c5f94ddc973c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:28.281328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf631dc-f994-4f0b-bee3-c5f94ddc973c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:28.382984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:28.383485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:28.383759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:28.384023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] PATH_INFO: `/v3/projects/3a4868eeca2540fa857ab5e60c705653` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:28.393757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4868eeca2540fa857ab5e60c705653)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:28.394781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:28.394781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Aug 24 09:59:28.394781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: Aug 24 09:59:28.395969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:28.452672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4868eeca2540fa857ab5e60c705653', 'request_id': 'req-f35a838c-13c7-4762-b0ba-4a3664fb7406'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:28.466901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4868eeca2540fa857ab5e60c705653', 'request_id': 'req-f35a838c-13c7-4762-b0ba-4a3664fb7406'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:28.472953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4868eeca2540fa857ab5e60c705653', 'request_id': 'req-f35a838c-13c7-4762-b0ba-4a3664fb7406'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:28.500614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] The token cache is being invalidate because project 3a4868eeca2540fa857ab5e60c705653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:59:28.500904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:28.501332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f35a838c-13c7-4762-b0ba-4a3664fb7406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:28.502769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 761/1510] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 09:59:28 2022] DELETE /identity/v3/projects/3a4868eeca2540fa857ab5e60c705653 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:37.931593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=62 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:38.498551 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:00.218696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030ea86-038f-4ba7-9bf0-e200dd35a67b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.219304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030ea86-038f-4ba7-9bf0-e200dd35a67b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.219304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030ea86-038f-4ba7-9bf0-e200dd35a67b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.220629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:00.225555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e60c3-defb-42da-9f33-179b1edd179a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.225991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e60c3-defb-42da-9f33-179b1edd179a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.226128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e60c3-defb-42da-9f33-179b1edd179a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.306900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b12e60c3-defb-42da-9f33-179b1edd179a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:00.308028 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b12e60c3-defb-42da-9f33-179b1edd179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:00.417554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:00.572194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030ea86-038f-4ba7-9bf0-e200dd35a67b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:00.574582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b030ea86-038f-4ba7-9bf0-e200dd35a67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:00.694068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030ea86-038f-4ba7-9bf0-e200dd35a67b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:00.699555 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49af7f-aae8-4c65-9511-694e030207ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:00.700184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49af7f-aae8-4c65-9511-694e030207ac service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.700469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49af7f-aae8-4c65-9511-694e030207ac service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.700746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49af7f-aae8-4c65-9511-694e030207ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.708859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49af7f-aae8-4c65-9511-694e030207ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:00.710457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49af7f-aae8-4c65-9511-694e030207ac service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:00.719787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 751/1513] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:01.562689 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d40319-685d-4f69-a045-621bb462f273 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:01.563263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d40319-685d-4f69-a045-621bb462f273 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:01.563263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d40319-685d-4f69-a045-621bb462f273 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:01.564693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:01.569644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfa7e3-3588-4b30-8ac4-8f57bd513ba7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:01.570082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfa7e3-3588-4b30-8ac4-8f57bd513ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:01.570185 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfa7e3-3588-4b30-8ac4-8f57bd513ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:01.589772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7fbfa7e3-3588-4b30-8ac4-8f57bd513ba7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:01.590976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7fbfa7e3-3588-4b30-8ac4-8f57bd513ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:01.599441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:01.749613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d40319-685d-4f69-a045-621bb462f273 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:01.764584 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d40319-685d-4f69-a045-621bb462f273 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:01.770739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc676cc-3428-4235-bbfc-ec3119f2a6ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:01.771246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc676cc-3428-4235-bbfc-ec3119f2a6ba service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:01.771528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc676cc-3428-4235-bbfc-ec3119f2a6ba service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:01.771804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc676cc-3428-4235-bbfc-ec3119f2a6ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:01.780499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc676cc-3428-4235-bbfc-ec3119f2a6ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:01.782207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc676cc-3428-4235-bbfc-ec3119f2a6ba service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:01.793364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 764/1516] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:02.731139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fa570-ce1e-4947-86f0-5a50b94379cf None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:02.731674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fa570-ce1e-4947-86f0-5a50b94379cf None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:02.731674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fa570-ce1e-4947-86f0-5a50b94379cf None None] PATH_INFO: `/v3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:02.732996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 753/1517] 10.222.0.73 () {54 vars in 925 bytes} [Wed Aug 24 10:01:02 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:02.756945 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ff692-68d7-495d-a358-ff47bc62017f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:02.757399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ff692-68d7-495d-a358-ff47bc62017f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:02.757538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ff692-68d7-495d-a358-ff47bc62017f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:02.817599 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-382ff692-68d7-495d-a358-ff47bc62017f None None] MFA Rules not processed for user `ef8d30e87c624d6180656d998afcebe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:02.818814 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-382ff692-68d7-495d-a358-ff47bc62017f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:02.835130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:02.889479 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 765/1518] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 24 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:06.724225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509f27d-7957-40fb-8c86-ff8f4b8b502e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:06.724757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509f27d-7957-40fb-8c86-ff8f4b8b502e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:06.724757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509f27d-7957-40fb-8c86-ff8f4b8b502e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:06.726041 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 754/1519] 10.222.0.86 () {54 vars in 961 bytes} [Wed Aug 24 10:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:06.731138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec045a99-6c6e-4ad0-aa06-00ae95984a02 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:06.731596 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec045a99-6c6e-4ad0-aa06-00ae95984a02 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:06.731712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec045a99-6c6e-4ad0-aa06-00ae95984a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:06.752276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ec045a99-6c6e-4ad0-aa06-00ae95984a02 None None] MFA Rules not processed for user `ef8d30e87c624d6180656d998afcebe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:06.753466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ec045a99-6c6e-4ad0-aa06-00ae95984a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:06.762016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 766/1520] 10.222.0.86 () {58 vars in 1058 bytes} [Wed Aug 24 10:01:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:06.769601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509f27d-7957-40fb-8c86-ff8f4b8b502e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:06.782752 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509f27d-7957-40fb-8c86-ff8f4b8b502e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:06.787842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412f3b2-744a-4dc2-acc2-99cc62cb2cfb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:06.788463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412f3b2-744a-4dc2-acc2-99cc62cb2cfb service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:06.788734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412f3b2-744a-4dc2-acc2-99cc62cb2cfb service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:06.789000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412f3b2-744a-4dc2-acc2-99cc62cb2cfb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:06.797814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412f3b2-744a-4dc2-acc2-99cc62cb2cfb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:06.799390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412f3b2-744a-4dc2-acc2-99cc62cb2cfb service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:06.808824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 755/1521] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:13.133983 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2ead0-6cf6-4963-8a18-f97a7532549b None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:13.134541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2ead0-6cf6-4963-8a18-f97a7532549b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:13.134541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2ead0-6cf6-4963-8a18-f97a7532549b None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:13.135785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:13.140687 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74123215-5398-4bd4-95f8-18fc015b5dab None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:13.141321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74123215-5398-4bd4-95f8-18fc015b5dab None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:13.141321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74123215-5398-4bd4-95f8-18fc015b5dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:13.160847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-74123215-5398-4bd4-95f8-18fc015b5dab None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:13.162030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-74123215-5398-4bd4-95f8-18fc015b5dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:13.169703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:13.320957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2ead0-6cf6-4963-8a18-f97a7532549b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:13.335875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2ead0-6cf6-4963-8a18-f97a7532549b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:13.341274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd034489-af7e-4dab-923b-366bd26c706a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:13.341859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034489-af7e-4dab-923b-366bd26c706a service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:13.342139 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034489-af7e-4dab-923b-366bd26c706a service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:13.342411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034489-af7e-4dab-923b-366bd26c706a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:13.351000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd034489-af7e-4dab-923b-366bd26c706a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:13.352619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd034489-af7e-4dab-923b-366bd26c706a service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:13.361834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 768/1524] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:14.179366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c42225-1efe-46fe-907d-21217e5e49ed None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.179911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c42225-1efe-46fe-907d-21217e5e49ed None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.179911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c42225-1efe-46fe-907d-21217e5e49ed None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.181278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 10:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:14.186014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2b9c0-465f-441b-9e50-e1881d7ab7f3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.186414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2b9c0-465f-441b-9e50-e1881d7ab7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.186567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2b9c0-465f-441b-9e50-e1881d7ab7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.206361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-78f2b9c0-465f-441b-9e50-e1881d7ab7f3 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:14.207593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-78f2b9c0-465f-441b-9e50-e1881d7ab7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:14.215513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 10:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:14.369207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c42225-1efe-46fe-907d-21217e5e49ed None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:14.383822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c42225-1efe-46fe-907d-21217e5e49ed None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:14.389206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf27627-cdaa-48dd-bed5-0c96955d4499 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:14.389785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf27627-cdaa-48dd-bed5-0c96955d4499 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.390115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf27627-cdaa-48dd-bed5-0c96955d4499 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.390358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf27627-cdaa-48dd-bed5-0c96955d4499 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.398340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf27627-cdaa-48dd-bed5-0c96955d4499 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:14.399906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf27627-cdaa-48dd-bed5-0c96955d4499 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:14.409635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 758/1527] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:18.448101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eeb7f0-d288-433b-932e-2b00d6006bc2 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.448706 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eeb7f0-d288-433b-932e-2b00d6006bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.448706 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eeb7f0-d288-433b-932e-2b00d6006bc2 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.450017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:18.455337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136186f6-926a-4815-b2ae-2b80e0b2412d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.455769 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136186f6-926a-4815-b2ae-2b80e0b2412d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.455912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136186f6-926a-4815-b2ae-2b80e0b2412d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.475276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-136186f6-926a-4815-b2ae-2b80e0b2412d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:18.476505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-136186f6-926a-4815-b2ae-2b80e0b2412d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:18.484959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:18.771998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eeb7f0-d288-433b-932e-2b00d6006bc2 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:18.773763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-26eeb7f0-d288-433b-932e-2b00d6006bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:18.854326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eeb7f0-d288-433b-932e-2b00d6006bc2 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:18.857934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536ff12-3b4a-4a70-80e1-1bdaf1c13feb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:18.858554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536ff12-3b4a-4a70-80e1-1bdaf1c13feb service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.858859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536ff12-3b4a-4a70-80e1-1bdaf1c13feb service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.859137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536ff12-3b4a-4a70-80e1-1bdaf1c13feb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.868406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536ff12-3b4a-4a70-80e1-1bdaf1c13feb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:18.869877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536ff12-3b4a-4a70-80e1-1bdaf1c13feb service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:18.879174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 771/1530] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:20.791995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38572e77-6f23-43c8-b187-231cf03c995e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:20.792512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38572e77-6f23-43c8-b187-231cf03c995e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:20.792630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38572e77-6f23-43c8-b187-231cf03c995e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:20.794021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:20.799551 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69883fc7-f18b-4766-8f26-6aa23f1933b8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:20.800010 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69883fc7-f18b-4766-8f26-6aa23f1933b8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:20.800157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69883fc7-f18b-4766-8f26-6aa23f1933b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:20.819167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-69883fc7-f18b-4766-8f26-6aa23f1933b8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:20.820410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-69883fc7-f18b-4766-8f26-6aa23f1933b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:20.828634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:21.119701 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38572e77-6f23-43c8-b187-231cf03c995e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:21.130621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38572e77-6f23-43c8-b187-231cf03c995e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:21.134225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a91f37-3734-4a51-891c-ced439bbe16d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:21.134814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a91f37-3734-4a51-891c-ced439bbe16d service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:21.135093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a91f37-3734-4a51-891c-ced439bbe16d service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:21.135361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a91f37-3734-4a51-891c-ced439bbe16d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:21.145135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a91f37-3734-4a51-891c-ced439bbe16d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:21.146563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a91f37-3734-4a51-891c-ced439bbe16d service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:21.156504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 761/1533] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:24.747229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ab42-7479-43da-b206-a8b70a444753 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:24.747803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ab42-7479-43da-b206-a8b70a444753 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:24.747803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ab42-7479-43da-b206-a8b70a444753 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:24.749515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:24.754462 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39319627-17e1-47a9-83b4-fa2f9f25020c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:24.754973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39319627-17e1-47a9-83b4-fa2f9f25020c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:24.754973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39319627-17e1-47a9-83b4-fa2f9f25020c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:24.774606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-39319627-17e1-47a9-83b4-fa2f9f25020c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:24.775800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-39319627-17e1-47a9-83b4-fa2f9f25020c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:24.785574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:25.347208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9ab42-7479-43da-b206-a8b70a444753 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:25.348911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-76a9ab42-7479-43da-b206-a8b70a444753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:25.416917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:25.431233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9ab42-7479-43da-b206-a8b70a444753 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:25.434911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a364f-7d95-4453-bdba-8436030e794b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:25.435506 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a364f-7d95-4453-bdba-8436030e794b service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:25.435811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a364f-7d95-4453-bdba-8436030e794b service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:25.436109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a364f-7d95-4453-bdba-8436030e794b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:25.445409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a364f-7d95-4453-bdba-8436030e794b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:25.446836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a364f-7d95-4453-bdba-8436030e794b service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:25.456619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 774/1536] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:27.355005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add14642-397d-4191-b56f-f1b0c5893630 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.355570 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add14642-397d-4191-b56f-f1b0c5893630 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.355570 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add14642-397d-4191-b56f-f1b0c5893630 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.357001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 10:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:27.363156 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687da4-f04e-4714-bbae-8e73ba3c01c8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.363603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687da4-f04e-4714-bbae-8e73ba3c01c8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.363702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687da4-f04e-4714-bbae-8e73ba3c01c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.382520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d8687da4-f04e-4714-bbae-8e73ba3c01c8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:27.383694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d8687da4-f04e-4714-bbae-8e73ba3c01c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:27.391522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 10:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:27.932647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add14642-397d-4191-b56f-f1b0c5893630 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.934793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-add14642-397d-4191-b56f-f1b0c5893630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:27.943479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=84 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:27.949893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add14642-397d-4191-b56f-f1b0c5893630 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.954339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3247ac-f233-48f5-bd5d-da0364a4839a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.954931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3247ac-f233-48f5-bd5d-da0364a4839a service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.955214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3247ac-f233-48f5-bd5d-da0364a4839a service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.955488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3247ac-f233-48f5-bd5d-da0364a4839a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.964528 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3247ac-f233-48f5-bd5d-da0364a4839a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.965956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3247ac-f233-48f5-bd5d-da0364a4839a service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.975734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 764/1539] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:37.377920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=11 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:37.971476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:52.888402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07133562-dace-4310-aa0a-ffbd69603487 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:52.889049 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07133562-dace-4310-aa0a-ffbd69603487 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:52.889049 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07133562-dace-4310-aa0a-ffbd69603487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:52.916149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-07133562-dace-4310-aa0a-ffbd69603487 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:52.917396 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-07133562-dace-4310-aa0a-ffbd69603487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:52.929754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 776/1540] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:52.938705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796213f2-a888-4f72-aa3b-8a42560d7a5a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:52.939776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796213f2-a888-4f72-aa3b-8a42560d7a5a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:52.940273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796213f2-a888-4f72-aa3b-8a42560d7a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:52.973155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-796213f2-a888-4f72-aa3b-8a42560d7a5a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:52.974755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-796213f2-a888-4f72-aa3b-8a42560d7a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:52.988920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 765/1541] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:52.993943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07133562-dace-4310-aa0a-ffbd69603487 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.010681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07133562-dace-4310-aa0a-ffbd69603487 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.022330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97929948-f27f-49d1-a871-568a36bc3445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.023500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97929948-f27f-49d1-a871-568a36bc3445 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.023931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97929948-f27f-49d1-a871-568a36bc3445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.024467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97929948-f27f-49d1-a871-568a36bc3445 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.025265 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c451b-3949-4ca9-b862-525da9507bd0 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.025495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97929948-f27f-49d1-a871-568a36bc3445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:53.025698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c451b-3949-4ca9-b862-525da9507bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.026052 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c451b-3949-4ca9-b862-525da9507bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.026297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97929948-f27f-49d1-a871-568a36bc3445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.028585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97929948-f27f-49d1-a871-568a36bc3445 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.043161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 777/1542] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:53.051857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97929948-f27f-49d1-a871-568a36bc3445 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.065879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-170c451b-3949-4ca9-b862-525da9507bd0 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.067485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97929948-f27f-49d1-a871-568a36bc3445 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.068025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-170c451b-3949-4ca9-b862-525da9507bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.084102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 766/1543] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:53.084326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29d4b9-e019-421b-b444-8cd247316d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.084909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d4b9-e019-421b-b444-8cd247316d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.085155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d4b9-e019-421b-b444-8cd247316d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.085433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d4b9-e019-421b-b444-8cd247316d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.086075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29d4b9-e019-421b-b444-8cd247316d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.087101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-9b29d4b9-e019-421b-b444-8cd247316d2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:02:53.087101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Aug 24 10:02:53.087101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: Aug 24 10:02:53.092581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29d4b9-e019-421b-b444-8cd247316d2d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.100346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850297a8-a8f6-4eb5-89ad-46a5e5006142 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.101012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850297a8-a8f6-4eb5-89ad-46a5e5006142 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.101525 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850297a8-a8f6-4eb5-89ad-46a5e5006142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.149922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-850297a8-a8f6-4eb5-89ad-46a5e5006142 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.151545 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-850297a8-a8f6-4eb5-89ad-46a5e5006142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.163892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 778/1544] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:53.165299 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e01ba4-f991-4522-899e-70d682ff5366 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.165595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e01ba4-f991-4522-899e-70d682ff5366 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.165873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e01ba4-f991-4522-899e-70d682ff5366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.181019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 767/1545] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:53.182254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850297a8-a8f6-4eb5-89ad-46a5e5006142 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.198387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850297a8-a8f6-4eb5-89ad-46a5e5006142 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.205252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d8e01ba4-f991-4522-899e-70d682ff5366 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.206512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d8e01ba4-f991-4522-899e-70d682ff5366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.302581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 779/1546] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.304009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e01ba4-f991-4522-899e-70d682ff5366 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.314463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc003209-6ce2-40d1-86c0-acd050727adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.315610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc003209-6ce2-40d1-86c0-acd050727adb admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.315885 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc003209-6ce2-40d1-86c0-acd050727adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.316176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc003209-6ce2-40d1-86c0-acd050727adb admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.316805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc003209-6ce2-40d1-86c0-acd050727adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.317823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-bc003209-6ce2-40d1-86c0-acd050727adb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:02:53.317823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 24 10:02:53.317823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: Aug 24 10:02:53.319161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc003209-6ce2-40d1-86c0-acd050727adb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.321031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e01ba4-f991-4522-899e-70d682ff5366 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.336770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7306ea-2418-4e1a-ab5b-1b787e795ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.337776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7306ea-2418-4e1a-ab5b-1b787e795ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.338313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7306ea-2418-4e1a-ab5b-1b787e795ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.338956 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7306ea-2418-4e1a-ab5b-1b787e795ecc admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.339852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7306ea-2418-4e1a-ab5b-1b787e795ecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:53.340406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7306ea-2418-4e1a-ab5b-1b787e795ecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.342685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7306ea-2418-4e1a-ab5b-1b787e795ecc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.364667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 768/1547] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:53.365750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e674285-bb70-4ce0-9ebd-aa4ae295de37 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.366036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e674285-bb70-4ce0-9ebd-aa4ae295de37 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.366403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e674285-bb70-4ce0-9ebd-aa4ae295de37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.367920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 780/1548] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:53.370226 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d624af-b168-4ee9-8bfe-4e0151c78374 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.370758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d624af-b168-4ee9-8bfe-4e0151c78374 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.371254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d624af-b168-4ee9-8bfe-4e0151c78374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.405195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5e674285-bb70-4ce0-9ebd-aa4ae295de37 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.406675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-29d624af-b168-4ee9-8bfe-4e0151c78374 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.406988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5e674285-bb70-4ce0-9ebd-aa4ae295de37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.408108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-29d624af-b168-4ee9-8bfe-4e0151c78374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.419112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 769/1549] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.420352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e674285-bb70-4ce0-9ebd-aa4ae295de37 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.424092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 781/1550] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.427993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db34b3b-0466-4859-81b5-8efb58a0c3b6 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.428356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db34b3b-0466-4859-81b5-8efb58a0c3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.428633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db34b3b-0466-4859-81b5-8efb58a0c3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.435842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e674285-bb70-4ce0-9ebd-aa4ae295de37 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.441225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce7864-2360-4674-905d-b770fe5541cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.441860 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce7864-2360-4674-905d-b770fe5541cd admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.442170 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce7864-2360-4674-905d-b770fe5541cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.442442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce7864-2360-4674-905d-b770fe5541cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.443114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce7864-2360-4674-905d-b770fe5541cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.444766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce7864-2360-4674-905d-b770fe5541cd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.452833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5db34b3b-0466-4859-81b5-8efb58a0c3b6 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.454074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5db34b3b-0466-4859-81b5-8efb58a0c3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.458991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 770/1551] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:53.458991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce7864-2360-4674-905d-b770fe5541cd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.463062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 782/1552] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.464356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dd83a-facd-45d9-aca1-06870b99cdfa None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.464645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dd83a-facd-45d9-aca1-06870b99cdfa None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.464926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dd83a-facd-45d9-aca1-06870b99cdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.467177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce7864-2360-4674-905d-b770fe5541cd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.472286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3d0e9-4863-4def-8179-88133bcc96ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.472880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3d0e9-4863-4def-8179-88133bcc96ea admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.473166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3d0e9-4863-4def-8179-88133bcc96ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.473441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3d0e9-4863-4def-8179-88133bcc96ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.474067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3d0e9-4863-4def-8179-88133bcc96ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.475973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3d0e9-4863-4def-8179-88133bcc96ea admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.485132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2c7dd83a-facd-45d9-aca1-06870b99cdfa None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.486333 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2c7dd83a-facd-45d9-aca1-06870b99cdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.495150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 783/1553] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.496517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cff35-1e43-4315-aa60-e0d062baf936 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.497053 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cff35-1e43-4315-aa60-e0d062baf936 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.497672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cff35-1e43-4315-aa60-e0d062baf936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.512986 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 771/1554] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:53.512986 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecbea8-aaf9-4536-8e4e-d5901c5accac None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.512986 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecbea8-aaf9-4536-8e4e-d5901c5accac None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.512986 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecbea8-aaf9-4536-8e4e-d5901c5accac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.522585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f51cff35-1e43-4315-aa60-e0d062baf936 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.523789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f51cff35-1e43-4315-aa60-e0d062baf936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.532179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-27ecbea8-aaf9-4536-8e4e-d5901c5accac None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.533474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-27ecbea8-aaf9-4536-8e4e-d5901c5accac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.584239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 784/1555] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.591000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cff35-1e43-4315-aa60-e0d062baf936 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.594013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 772/1556] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.595589 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b6b18-82b0-4360-a459-d1ac51a6ce4d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.595916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b6b18-82b0-4360-a459-d1ac51a6ce4d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.596484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b6b18-82b0-4360-a459-d1ac51a6ce4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.601463 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cff35-1e43-4315-aa60-e0d062baf936 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.606072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682dced-01b9-4711-817e-e0247070a924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.606670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682dced-01b9-4711-817e-e0247070a924 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.606949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682dced-01b9-4711-817e-e0247070a924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.607223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682dced-01b9-4711-817e-e0247070a924 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.607869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682dced-01b9-4711-817e-e0247070a924 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:53.608204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682dced-01b9-4711-817e-e0247070a924 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.609805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682dced-01b9-4711-817e-e0247070a924 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.617162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-875b6b18-82b0-4360-a459-d1ac51a6ce4d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.618807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-875b6b18-82b0-4360-a459-d1ac51a6ce4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.622259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 785/1557] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:53.623338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682dced-01b9-4711-817e-e0247070a924 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.628486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 773/1558] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.629928 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c3b12-c570-41d0-a932-88998c9fb943 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.630222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c3b12-c570-41d0-a932-88998c9fb943 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.630516 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c3b12-c570-41d0-a932-88998c9fb943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.634305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:53.638450 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682dced-01b9-4711-817e-e0247070a924 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.645359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6fcc5-5a16-40da-a30c-d42f371ddc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.646135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6fcc5-5a16-40da-a30c-d42f371ddc65 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.646416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6fcc5-5a16-40da-a30c-d42f371ddc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.646681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6fcc5-5a16-40da-a30c-d42f371ddc65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.647318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6fcc5-5a16-40da-a30c-d42f371ddc65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.648962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6fcc5-5a16-40da-a30c-d42f371ddc65 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.650465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-909c3b12-c570-41d0-a932-88998c9fb943 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.651840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-909c3b12-c570-41d0-a932-88998c9fb943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.663238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 786/1559] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:53.663394 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 774/1560] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.663394 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c3b12-c570-41d0-a932-88998c9fb943 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.663603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01121a-0aa4-4056-b764-21210688cfb5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.663603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01121a-0aa4-4056-b764-21210688cfb5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.663603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01121a-0aa4-4056-b764-21210688cfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.672767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c3b12-c570-41d0-a932-88998c9fb943 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.678042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00d451-135e-477a-b1fe-98366e663094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.678716 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00d451-135e-477a-b1fe-98366e663094 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.679091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00d451-135e-477a-b1fe-98366e663094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.679434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00d451-135e-477a-b1fe-98366e663094 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.680196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00d451-135e-477a-b1fe-98366e663094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.682059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ae01121a-0aa4-4056-b764-21210688cfb5 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.682184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00d451-135e-477a-b1fe-98366e663094 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.683633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ae01121a-0aa4-4056-b764-21210688cfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.691663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 787/1561] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.695643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7505561-fd20-47c1-985b-82f82558725e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.695643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7505561-fd20-47c1-985b-82f82558725e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.695643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7505561-fd20-47c1-985b-82f82558725e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.704082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:53.710309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 775/1562] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:53.710970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00d451-135e-477a-b1fe-98366e663094 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.714501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c7505561-fd20-47c1-985b-82f82558725e None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.715698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c7505561-fd20-47c1-985b-82f82558725e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.722738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00d451-135e-477a-b1fe-98366e663094 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.723836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 788/1563] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.728031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f8be1-beb4-4286-8bf4-f8492c0df891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.728227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7505561-fd20-47c1-985b-82f82558725e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.728669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f8be1-beb4-4286-8bf4-f8492c0df891 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.728950 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f8be1-beb4-4286-8bf4-f8492c0df891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.729227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f8be1-beb4-4286-8bf4-f8492c0df891 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.729900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f8be1-beb4-4286-8bf4-f8492c0df891 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:53.730182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f8be1-beb4-4286-8bf4-f8492c0df891 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.731199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-043f8be1-beb4-4286-8bf4-f8492c0df891 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:02:53.731199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 24 10:02:53.731199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: Aug 24 10:02:53.732119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f8be1-beb4-4286-8bf4-f8492c0df891 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.738186 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7505561-fd20-47c1-985b-82f82558725e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.743856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09b999-9379-4240-9fe9-302dd5e0ffdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.744633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09b999-9379-4240-9fe9-302dd5e0ffdc admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.745045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09b999-9379-4240-9fe9-302dd5e0ffdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.745334 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09b999-9379-4240-9fe9-302dd5e0ffdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.746201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09b999-9379-4240-9fe9-302dd5e0ffdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.747607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 776/1564] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:53.748239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09b999-9379-4240-9fe9-302dd5e0ffdc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.748972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701527c-8298-4de2-b1d6-ccd008d080f6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.749262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701527c-8298-4de2-b1d6-ccd008d080f6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.749539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701527c-8298-4de2-b1d6-ccd008d080f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.775892 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2701527c-8298-4de2-b1d6-ccd008d080f6 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:53.777364 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2701527c-8298-4de2-b1d6-ccd008d080f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.786240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 789/1565] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:53.786240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09b999-9379-4240-9fe9-302dd5e0ffdc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.788861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 777/1566] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:53.789739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701527c-8298-4de2-b1d6-ccd008d080f6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.801224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701527c-8298-4de2-b1d6-ccd008d080f6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.852195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09b999-9379-4240-9fe9-302dd5e0ffdc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.857155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d3458-4981-4352-9999-84451721f13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.857814 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d3458-4981-4352-9999-84451721f13e admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.858040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22722cd-8df6-4eff-88d6-b33c5af5b435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.858244 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d3458-4981-4352-9999-84451721f13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.858507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d3458-4981-4352-9999-84451721f13e admin admin] PATH_INFO: `/v3/projects/abcce51c445b46fdab781056f45743cd/users/ca01a580c6724762b61440711b94fbf7/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.858631 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22722cd-8df6-4eff-88d6-b33c5af5b435 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.859035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22722cd-8df6-4eff-88d6-b33c5af5b435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.859312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22722cd-8df6-4eff-88d6-b33c5af5b435 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.859956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22722cd-8df6-4eff-88d6-b33c5af5b435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:53.860289 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22722cd-8df6-4eff-88d6-b33c5af5b435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.862639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22722cd-8df6-4eff-88d6-b33c5af5b435 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.882658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 778/1567] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:53.882902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d3458-4981-4352-9999-84451721f13e admin admin] RBAC: Authorizing `identity:create_grant(project_id=abcce51c445b46fdab781056f45743cd, user_id=ca01a580c6724762b61440711b94fbf7, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.883054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22722cd-8df6-4eff-88d6-b33c5af5b435 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.883165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-c97d3458-4981-4352-9999-84451721f13e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:02:53.883165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Aug 24 10:02:53.883165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: Aug 24 10:02:53.883165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d3458-4981-4352-9999-84451721f13e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.889535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22722cd-8df6-4eff-88d6-b33c5af5b435 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.892655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 790/1568] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:53 2022] PUT /identity/v3/projects/abcce51c445b46fdab781056f45743cd/users/ca01a580c6724762b61440711b94fbf7/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:53.893595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d3458-4981-4352-9999-84451721f13e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.947117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629a448-9639-482e-8564-681c04438637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.947802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629a448-9639-482e-8564-681c04438637 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.948000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629a448-9639-482e-8564-681c04438637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.948310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629a448-9639-482e-8564-681c04438637 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.949008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629a448-9639-482e-8564-681c04438637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:53.949321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629a448-9639-482e-8564-681c04438637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.950922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629a448-9639-482e-8564-681c04438637 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.963757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 779/1569] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:53.964510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d3458-4981-4352-9999-84451721f13e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.969701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce3eba-cefa-4882-9625-a2bd80cee04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.970313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce3eba-cefa-4882-9625-a2bd80cee04e admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.970652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce3eba-cefa-4882-9625-a2bd80cee04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.971453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce3eba-cefa-4882-9625-a2bd80cee04e admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.972269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce3eba-cefa-4882-9625-a2bd80cee04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.972402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629a448-9639-482e-8564-681c04438637 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.973900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce3eba-cefa-4882-9625-a2bd80cee04e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.982011 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629a448-9639-482e-8564-681c04438637 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.988383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 791/1570] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:53.988383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce3eba-cefa-4882-9625-a2bd80cee04e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.988634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b03272-39e2-455b-87e3-32a7208b4026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.988634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b03272-39e2-455b-87e3-32a7208b4026 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.988634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b03272-39e2-455b-87e3-32a7208b4026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.988634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b03272-39e2-455b-87e3-32a7208b4026 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.989138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b03272-39e2-455b-87e3-32a7208b4026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:53.989436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b03272-39e2-455b-87e3-32a7208b4026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.991013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b03272-39e2-455b-87e3-32a7208b4026 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.996032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce3eba-cefa-4882-9625-a2bd80cee04e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.001266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcfafb-6e2b-49ec-91d1-23e91a504f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.001796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcfafb-6e2b-49ec-91d1-23e91a504f17 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.002079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcfafb-6e2b-49ec-91d1-23e91a504f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.002356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcfafb-6e2b-49ec-91d1-23e91a504f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.003009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffcfafb-6e2b-49ec-91d1-23e91a504f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.003112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 780/1571] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:54.003867 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b03272-39e2-455b-87e3-32a7208b4026 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.004959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffcfafb-6e2b-49ec-91d1-23e91a504f17 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.013596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b03272-39e2-455b-87e3-32a7208b4026 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.018358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2512ec7-f2c3-441f-ac8f-e8dd9cd22957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.018957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2512ec7-f2c3-441f-ac8f-e8dd9cd22957 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.019238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2512ec7-f2c3-441f-ac8f-e8dd9cd22957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.019510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2512ec7-f2c3-441f-ac8f-e8dd9cd22957 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.020504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2512ec7-f2c3-441f-ac8f-e8dd9cd22957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.022429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2512ec7-f2c3-441f-ac8f-e8dd9cd22957 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.034524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 792/1572] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:53 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.035761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcfafb-6e2b-49ec-91d1-23e91a504f17 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.047178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 781/1573] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.047884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2512ec7-f2c3-441f-ac8f-e8dd9cd22957 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.055154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:54.059719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2512ec7-f2c3-441f-ac8f-e8dd9cd22957 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.097433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:54.101970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcfafb-6e2b-49ec-91d1-23e91a504f17 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.107104 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9105e-6665-464a-b4f1-ea91f1137b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.107710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9105e-6665-464a-b4f1-ea91f1137b39 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.107987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9105e-6665-464a-b4f1-ea91f1137b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.108320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9105e-6665-464a-b4f1-ea91f1137b39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.108949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9105e-6665-464a-b4f1-ea91f1137b39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:54.109255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9105e-6665-464a-b4f1-ea91f1137b39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.110882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9105e-6665-464a-b4f1-ea91f1137b39 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.129854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 793/1574] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:54.130061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e3a3e-90de-4333-9ff3-71aecc1ea319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.130061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e3a3e-90de-4333-9ff3-71aecc1ea319 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.130289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9231a6-4b66-4faf-9d65-695ca0835664 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.130381 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e3a3e-90de-4333-9ff3-71aecc1ea319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.130467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9231a6-4b66-4faf-9d65-695ca0835664 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.130564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e3a3e-90de-4333-9ff3-71aecc1ea319 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.130650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9231a6-4b66-4faf-9d65-695ca0835664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.130744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e3a3e-90de-4333-9ff3-71aecc1ea319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.130744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e3a3e-90de-4333-9ff3-71aecc1ea319 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.152512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8e9231a6-4b66-4faf-9d65-695ca0835664 None None] MFA Rules not processed for user `ca01a580c6724762b61440711b94fbf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:54.153734 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8e9231a6-4b66-4faf-9d65-695ca0835664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:54.155381 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 782/1575] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.156182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e3a3e-90de-4333-9ff3-71aecc1ea319 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.219480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e3a3e-90de-4333-9ff3-71aecc1ea319 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.223195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 794/1576] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:54.224035 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9231a6-4b66-4faf-9d65-695ca0835664 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.228136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf520d7-aa32-4615-a298-9ff4c204809d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.228668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf520d7-aa32-4615-a298-9ff4c204809d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.228948 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf520d7-aa32-4615-a298-9ff4c204809d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.229220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf520d7-aa32-4615-a298-9ff4c204809d admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.229960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf520d7-aa32-4615-a298-9ff4c204809d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.231939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf520d7-aa32-4615-a298-9ff4c204809d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.237350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9231a6-4b66-4faf-9d65-695ca0835664 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.242658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed518d9-b75b-433d-b688-e2f0dfbb5819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.243728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed518d9-b75b-433d-b688-e2f0dfbb5819 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.243728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed518d9-b75b-433d-b688-e2f0dfbb5819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.244012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed518d9-b75b-433d-b688-e2f0dfbb5819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.244859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed518d9-b75b-433d-b688-e2f0dfbb5819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.246732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed518d9-b75b-433d-b688-e2f0dfbb5819 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.259278 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 795/1577] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:54.260644 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed518d9-b75b-433d-b688-e2f0dfbb5819 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.266196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 783/1578] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.266196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf520d7-aa32-4615-a298-9ff4c204809d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.272662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed518d9-b75b-433d-b688-e2f0dfbb5819 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.328109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf520d7-aa32-4615-a298-9ff4c204809d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.331510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46394e-4799-4c42-b676-809a65c4401c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.332133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46394e-4799-4c42-b676-809a65c4401c admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.332411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46394e-4799-4c42-b676-809a65c4401c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.332674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46394e-4799-4c42-b676-809a65c4401c admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.332974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1069f42-b59e-446f-bf61-5acfb6c4c626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.333287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46394e-4799-4c42-b676-809a65c4401c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.333571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1069f42-b59e-446f-bf61-5acfb6c4c626 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.333848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1069f42-b59e-446f-bf61-5acfb6c4c626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.334117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1069f42-b59e-446f-bf61-5acfb6c4c626 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.334933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1069f42-b59e-446f-bf61-5acfb6c4c626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.335191 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46394e-4799-4c42-b676-809a65c4401c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.338484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1069f42-b59e-446f-bf61-5acfb6c4c626 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.362983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 784/1579] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.363874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1069f42-b59e-446f-bf61-5acfb6c4c626 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.366852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 796/1580] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.367804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46394e-4799-4c42-b676-809a65c4401c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.376511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1069f42-b59e-446f-bf61-5acfb6c4c626 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.380992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:54.412649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:54.432453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46394e-4799-4c42-b676-809a65c4401c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.436700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8a368-98c1-43ba-a20a-8457da3f65fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.437037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79ee6b-81ec-4d7f-9f25-d7b91356f4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.437307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8a368-98c1-43ba-a20a-8457da3f65fd admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.437589 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8a368-98c1-43ba-a20a-8457da3f65fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.437699 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79ee6b-81ec-4d7f-9f25-d7b91356f4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.437862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8a368-98c1-43ba-a20a-8457da3f65fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.437975 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79ee6b-81ec-4d7f-9f25-d7b91356f4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.438141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79ee6b-81ec-4d7f-9f25-d7b91356f4f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.438505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8a368-98c1-43ba-a20a-8457da3f65fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.438755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79ee6b-81ec-4d7f-9f25-d7b91356f4f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.440177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8a368-98c1-43ba-a20a-8457da3f65fd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.440738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79ee6b-81ec-4d7f-9f25-d7b91356f4f9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.449976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 785/1581] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:54.450850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8a368-98c1-43ba-a20a-8457da3f65fd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.461305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8a368-98c1-43ba-a20a-8457da3f65fd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.466101 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53f0a9-92f7-440e-bdb2-9099e517956a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.466690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53f0a9-92f7-440e-bdb2-9099e517956a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.466969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53f0a9-92f7-440e-bdb2-9099e517956a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.467245 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53f0a9-92f7-440e-bdb2-9099e517956a admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.467964 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53f0a9-92f7-440e-bdb2-9099e517956a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.469827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 797/1582] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.469956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53f0a9-92f7-440e-bdb2-9099e517956a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.470673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79ee6b-81ec-4d7f-9f25-d7b91356f4f9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.492672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 786/1583] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.493608 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53f0a9-92f7-440e-bdb2-9099e517956a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.503415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53f0a9-92f7-440e-bdb2-9099e517956a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.534282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79ee6b-81ec-4d7f-9f25-d7b91356f4f9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.538900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0c288-00eb-4fa9-8f47-46ba89695127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.539496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0c288-00eb-4fa9-8f47-46ba89695127 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.539775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0c288-00eb-4fa9-8f47-46ba89695127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.540077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0c288-00eb-4fa9-8f47-46ba89695127 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.540830 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0c288-00eb-4fa9-8f47-46ba89695127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.542776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0c288-00eb-4fa9-8f47-46ba89695127 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.559856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab2724-5117-46e2-a518-691180a4f136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.560596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab2724-5117-46e2-a518-691180a4f136 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.560895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab2724-5117-46e2-a518-691180a4f136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.561116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab2724-5117-46e2-a518-691180a4f136 admin admin] PATH_INFO: `/v3/projects/024d85dd05ad4caf8cf52efe7940ba0f/users/48f76882fbf8462d976d5e5c5cbd4571/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.571845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 798/1584] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.572761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0c288-00eb-4fa9-8f47-46ba89695127 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.577310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab2724-5117-46e2-a518-691180a4f136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=024d85dd05ad4caf8cf52efe7940ba0f, user_id=48f76882fbf8462d976d5e5c5cbd4571, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.581487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab2724-5117-46e2-a518-691180a4f136 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.596607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 787/1585] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:54 2022] PUT /identity/v3/projects/024d85dd05ad4caf8cf52efe7940ba0f/users/48f76882fbf8462d976d5e5c5cbd4571/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:54.597453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab2724-5117-46e2-a518-691180a4f136 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.637271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0c288-00eb-4fa9-8f47-46ba89695127 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.667648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:54.671667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab2724-5117-46e2-a518-691180a4f136 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.677457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec791b-7d76-4dbf-a5dd-c4a2af5efb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.678061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec791b-7d76-4dbf-a5dd-c4a2af5efb99 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.678337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec791b-7d76-4dbf-a5dd-c4a2af5efb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.678635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec791b-7d76-4dbf-a5dd-c4a2af5efb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.679287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec791b-7d76-4dbf-a5dd-c4a2af5efb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.681089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec791b-7d76-4dbf-a5dd-c4a2af5efb99 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.695224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 788/1586] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:54.696342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec791b-7d76-4dbf-a5dd-c4a2af5efb99 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.702980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163d718-6d6e-446f-a9c9-f073bbc9a9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.703682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163d718-6d6e-446f-a9c9-f073bbc9a9bc admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.703967 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163d718-6d6e-446f-a9c9-f073bbc9a9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.704272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163d718-6d6e-446f-a9c9-f073bbc9a9bc admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.704905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163d718-6d6e-446f-a9c9-f073bbc9a9bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.707302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec791b-7d76-4dbf-a5dd-c4a2af5efb99 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.707443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163d718-6d6e-446f-a9c9-f073bbc9a9bc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.712099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b2d4f-3ba9-42d6-8272-36ecbb61b5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.712721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2d4f-3ba9-42d6-8272-36ecbb61b5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.713005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2d4f-3ba9-42d6-8272-36ecbb61b5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.713337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2d4f-3ba9-42d6-8272-36ecbb61b5d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.713956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b2d4f-3ba9-42d6-8272-36ecbb61b5d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.715879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b2d4f-3ba9-42d6-8272-36ecbb61b5d6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.730527 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 799/1587] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.731239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163d718-6d6e-446f-a9c9-f073bbc9a9bc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.739539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 789/1588] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.740613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b2d4f-3ba9-42d6-8272-36ecbb61b5d6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.741291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163d718-6d6e-446f-a9c9-f073bbc9a9bc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.745933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d873b3-9f8d-454f-ae6a-ee7103935d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.746529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d873b3-9f8d-454f-ae6a-ee7103935d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.746808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d873b3-9f8d-454f-ae6a-ee7103935d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.747080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d873b3-9f8d-454f-ae6a-ee7103935d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.747724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d873b3-9f8d-454f-ae6a-ee7103935d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.749367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d873b3-9f8d-454f-ae6a-ee7103935d0c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.751891 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b2d4f-3ba9-42d6-8272-36ecbb61b5d6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.757153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cb0a7-0687-4dda-b0a9-7923aa02d8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.757768 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cb0a7-0687-4dda-b0a9-7923aa02d8e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.758075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cb0a7-0687-4dda-b0a9-7923aa02d8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.758350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cb0a7-0687-4dda-b0a9-7923aa02d8e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.758967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cb0a7-0687-4dda-b0a9-7923aa02d8e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.759830 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 800/1589] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:54.760643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d873b3-9f8d-454f-ae6a-ee7103935d0c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.761048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cb0a7-0687-4dda-b0a9-7923aa02d8e5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.771371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d873b3-9f8d-454f-ae6a-ee7103935d0c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.777238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71bd94-d069-40fc-866d-b776e89cc7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.778217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71bd94-d069-40fc-866d-b776e89cc7f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.778521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71bd94-d069-40fc-866d-b776e89cc7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.778776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71bd94-d069-40fc-866d-b776e89cc7f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.779680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71bd94-d069-40fc-866d-b776e89cc7f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.780993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:54.781323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71bd94-d069-40fc-866d-b776e89cc7f9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.793354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 790/1590] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/users => generated 293 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.793672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 801/1591] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:54.794161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cb0a7-0687-4dda-b0a9-7923aa02d8e5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.794975 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71a35d-8ae9-4c6f-95c6-b4ef1399f659 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.795263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71a35d-8ae9-4c6f-95c6-b4ef1399f659 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.795540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71a35d-8ae9-4c6f-95c6-b4ef1399f659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.804454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cb0a7-0687-4dda-b0a9-7923aa02d8e5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.809039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50082d-a40e-4d74-b16e-759114236e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.809634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50082d-a40e-4d74-b16e-759114236e42 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.809913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50082d-a40e-4d74-b16e-759114236e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.810184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50082d-a40e-4d74-b16e-759114236e42 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.810800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50082d-a40e-4d74-b16e-759114236e42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.812741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50082d-a40e-4d74-b16e-759114236e42 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.823931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4b71a35d-8ae9-4c6f-95c6-b4ef1399f659 None None] MFA Rules not processed for user `48f76882fbf8462d976d5e5c5cbd4571`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:54.825399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4b71a35d-8ae9-4c6f-95c6-b4ef1399f659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:54.835506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 791/1592] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/users => generated 296 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:54.836347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50082d-a40e-4d74-b16e-759114236e42 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.847560 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50082d-a40e-4d74-b16e-759114236e42 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.852152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7eb19-a3fe-4474-aa02-ab4b5d4c54ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.852734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7eb19-a3fe-4474-aa02-ab4b5d4c54ed admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.853013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7eb19-a3fe-4474-aa02-ab4b5d4c54ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.853285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7eb19-a3fe-4474-aa02-ab4b5d4c54ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.853916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7eb19-a3fe-4474-aa02-ab4b5d4c54ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.855503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7eb19-a3fe-4474-aa02-ab4b5d4c54ed admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.865627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 792/1593] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:54.866420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7eb19-a3fe-4474-aa02-ab4b5d4c54ed admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.875569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7eb19-a3fe-4474-aa02-ab4b5d4c54ed admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.880820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23db5e-0e33-45d5-a263-7b0eda61d1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.881516 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23db5e-0e33-45d5-a263-7b0eda61d1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.881879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23db5e-0e33-45d5-a263-7b0eda61d1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.882224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23db5e-0e33-45d5-a263-7b0eda61d1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.882919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a23db5e-0e33-45d5-a263-7b0eda61d1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.884627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a23db5e-0e33-45d5-a263-7b0eda61d1d8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.893966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 802/1594] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:54.894885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71a35d-8ae9-4c6f-95c6-b4ef1399f659 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.898424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 793/1595] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:54.898424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23db5e-0e33-45d5-a263-7b0eda61d1d8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.907442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23db5e-0e33-45d5-a263-7b0eda61d1d8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.907589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71a35d-8ae9-4c6f-95c6-b4ef1399f659 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.912027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be94ed-41be-49bf-b5a0-82d6c0d8f642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.912235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbba744-ef28-4120-a369-88b38288d790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.912688 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be94ed-41be-49bf-b5a0-82d6c0d8f642 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.912796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbba744-ef28-4120-a369-88b38288d790 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.912970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be94ed-41be-49bf-b5a0-82d6c0d8f642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.913076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbba744-ef28-4120-a369-88b38288d790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.913244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be94ed-41be-49bf-b5a0-82d6c0d8f642 admin admin] PATH_INFO: `/v3/projects/d269b3486e3b4375800858b532f78f35/users/cb051667aa484eafa4fa6b66c15d649e/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.913348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbba744-ef28-4120-a369-88b38288d790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.913937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbba744-ef28-4120-a369-88b38288d790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.915538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbba744-ef28-4120-a369-88b38288d790 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.925521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 803/1596] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:54.926330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbba744-ef28-4120-a369-88b38288d790 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.931833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be94ed-41be-49bf-b5a0-82d6c0d8f642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d269b3486e3b4375800858b532f78f35, user_id=cb051667aa484eafa4fa6b66c15d649e, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.934612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be94ed-41be-49bf-b5a0-82d6c0d8f642 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.936227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbba744-ef28-4120-a369-88b38288d790 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.941842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ede63-5100-4e63-91a3-ec8b8113ff85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.942529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ede63-5100-4e63-91a3-ec8b8113ff85 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.942894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ede63-5100-4e63-91a3-ec8b8113ff85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.943231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ede63-5100-4e63-91a3-ec8b8113ff85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.943952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ede63-5100-4e63-91a3-ec8b8113ff85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.945650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ede63-5100-4e63-91a3-ec8b8113ff85 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.952089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 794/1597] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:54 2022] PUT /identity/v3/projects/d269b3486e3b4375800858b532f78f35/users/cb051667aa484eafa4fa6b66c15d649e/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:54.952089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be94ed-41be-49bf-b5a0-82d6c0d8f642 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.960011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 804/1598] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:54.960011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ede63-5100-4e63-91a3-ec8b8113ff85 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.971749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ede63-5100-4e63-91a3-ec8b8113ff85 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.031733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be94ed-41be-49bf-b5a0-82d6c0d8f642 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.039104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf511a-7052-499a-8572-b18a1622474c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.039890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29175a6e-2d7d-4240-a790-f71a25f58141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.040078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf511a-7052-499a-8572-b18a1622474c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.040170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29175a6e-2d7d-4240-a790-f71a25f58141 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.040247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf511a-7052-499a-8572-b18a1622474c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.040336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29175a6e-2d7d-4240-a790-f71a25f58141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.040426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf511a-7052-499a-8572-b18a1622474c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.040606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29175a6e-2d7d-4240-a790-f71a25f58141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.041045 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf511a-7052-499a-8572-b18a1622474c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.041227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29175a6e-2d7d-4240-a790-f71a25f58141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.042671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf511a-7052-499a-8572-b18a1622474c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.043092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29175a6e-2d7d-4240-a790-f71a25f58141 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.054921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 795/1599] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:55.055757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf511a-7052-499a-8572-b18a1622474c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.056222 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 805/1600] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:55.057111 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29175a6e-2d7d-4240-a790-f71a25f58141 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.062854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:55.069330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29175a6e-2d7d-4240-a790-f71a25f58141 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.071472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf511a-7052-499a-8572-b18a1622474c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.075667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd68fbb-5233-455f-8238-1d78e81710f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.076464 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd68fbb-5233-455f-8238-1d78e81710f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.076847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd68fbb-5233-455f-8238-1d78e81710f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.077241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd68fbb-5233-455f-8238-1d78e81710f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.078058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9e438-00a3-45a3-9fd6-4317d5cbd6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.078204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd68fbb-5233-455f-8238-1d78e81710f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.078989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9e438-00a3-45a3-9fd6-4317d5cbd6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.079414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9e438-00a3-45a3-9fd6-4317d5cbd6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.079848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9e438-00a3-45a3-9fd6-4317d5cbd6fc admin admin] PATH_INFO: `/v3/projects/7c1ad7a7198d4e78b9d03f00ce65e091/users/df9b402589fa4020bc241a4b9c06dd91/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.081118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd68fbb-5233-455f-8238-1d78e81710f1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.098802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9e438-00a3-45a3-9fd6-4317d5cbd6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c1ad7a7198d4e78b9d03f00ce65e091, user_id=df9b402589fa4020bc241a4b9c06dd91, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.102142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9e438-00a3-45a3-9fd6-4317d5cbd6fc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.116938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 806/1601] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:55.117910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd68fbb-5233-455f-8238-1d78e81710f1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.118023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 796/1602] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:55 2022] PUT /identity/v3/projects/7c1ad7a7198d4e78b9d03f00ce65e091/users/df9b402589fa4020bc241a4b9c06dd91/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:55.118804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9e438-00a3-45a3-9fd6-4317d5cbd6fc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.185425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd68fbb-5233-455f-8238-1d78e81710f1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.190676 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c802faa-aebd-4ebc-a97b-fca132d2319f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.191286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c802faa-aebd-4ebc-a97b-fca132d2319f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.191569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c802faa-aebd-4ebc-a97b-fca132d2319f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.191848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c802faa-aebd-4ebc-a97b-fca132d2319f admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.192212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9e438-00a3-45a3-9fd6-4317d5cbd6fc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.192539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c802faa-aebd-4ebc-a97b-fca132d2319f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.194144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c802faa-aebd-4ebc-a97b-fca132d2319f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.196908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2719cf-afa3-4b7a-bc5b-ea32882c1329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.197507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2719cf-afa3-4b7a-bc5b-ea32882c1329 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.197805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2719cf-afa3-4b7a-bc5b-ea32882c1329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.198075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2719cf-afa3-4b7a-bc5b-ea32882c1329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.198727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2719cf-afa3-4b7a-bc5b-ea32882c1329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.200472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2719cf-afa3-4b7a-bc5b-ea32882c1329 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.206767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 807/1603] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:55.207664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c802faa-aebd-4ebc-a97b-fca132d2319f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.210885 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 797/1604] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:55.213213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c71cdb-fa1f-4545-98cc-7d2e18dd6487 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.213213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c71cdb-fa1f-4545-98cc-7d2e18dd6487 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.213213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c71cdb-fa1f-4545-98cc-7d2e18dd6487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.220968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:55.222509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c802faa-aebd-4ebc-a97b-fca132d2319f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.228852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128a589-0948-4474-9d3d-0b497b6e642c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.229814 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128a589-0948-4474-9d3d-0b497b6e642c admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.230122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128a589-0948-4474-9d3d-0b497b6e642c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.230409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128a589-0948-4474-9d3d-0b497b6e642c admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.231055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128a589-0948-4474-9d3d-0b497b6e642c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.232908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128a589-0948-4474-9d3d-0b497b6e642c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.244304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f5c71cdb-fa1f-4545-98cc-7d2e18dd6487 None None] MFA Rules not processed for user `cb051667aa484eafa4fa6b66c15d649e`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:55.245105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f5c71cdb-fa1f-4545-98cc-7d2e18dd6487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:55.249797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 808/1605] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:55.250656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128a589-0948-4474-9d3d-0b497b6e642c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.262584 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128a589-0948-4474-9d3d-0b497b6e642c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.267282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bbe67-2d0a-49ca-9750-9e4e2fac19d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.267961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bbe67-2d0a-49ca-9750-9e4e2fac19d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.268412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bbe67-2d0a-49ca-9750-9e4e2fac19d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.268933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bbe67-2d0a-49ca-9750-9e4e2fac19d5 admin admin] PATH_INFO: `/v3/projects/bcf7710287bd41e99028493910906a24/users/da187b84e25a4a369aacfba4c5a9aa60/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.285851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bbe67-2d0a-49ca-9750-9e4e2fac19d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcf7710287bd41e99028493910906a24, user_id=da187b84e25a4a369aacfba4c5a9aa60, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.288498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bbe67-2d0a-49ca-9750-9e4e2fac19d5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.303241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 809/1606] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:55 2022] PUT /identity/v3/projects/bcf7710287bd41e99028493910906a24/users/da187b84e25a4a369aacfba4c5a9aa60/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:55.304267 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bbe67-2d0a-49ca-9750-9e4e2fac19d5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.319946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 798/1607] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:55.321356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c71cdb-fa1f-4545-98cc-7d2e18dd6487 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.332360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c71cdb-fa1f-4545-98cc-7d2e18dd6487 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.374425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bbe67-2d0a-49ca-9750-9e4e2fac19d5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.383136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb3bce-14de-45b8-9a43-3090b95c1656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.383837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb3bce-14de-45b8-9a43-3090b95c1656 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.384144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb3bce-14de-45b8-9a43-3090b95c1656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.384427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb3bce-14de-45b8-9a43-3090b95c1656 admin admin] PATH_INFO: `/v3/projects/1bdfd8ab42f9496d87a8cfcf78d7edc3/users/46c01f8b99e24be5b6c2d73b9b1bd9b8/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.395750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c2ae8-9a86-4fc9-89be-d6119764538c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.396392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c2ae8-9a86-4fc9-89be-d6119764538c admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.396675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c2ae8-9a86-4fc9-89be-d6119764538c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.396948 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c2ae8-9a86-4fc9-89be-d6119764538c admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.397578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c2ae8-9a86-4fc9-89be-d6119764538c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.399608 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c2ae8-9a86-4fc9-89be-d6119764538c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.400214 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb3bce-14de-45b8-9a43-3090b95c1656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bdfd8ab42f9496d87a8cfcf78d7edc3, user_id=46c01f8b99e24be5b6c2d73b9b1bd9b8, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.402903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb3bce-14de-45b8-9a43-3090b95c1656 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.417185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 810/1608] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:55 2022] PUT /identity/v3/projects/1bdfd8ab42f9496d87a8cfcf78d7edc3/users/46c01f8b99e24be5b6c2d73b9b1bd9b8/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:55.418557 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadb704-af88-407e-b90b-33374bd7ff08 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.418830 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadb704-af88-407e-b90b-33374bd7ff08 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.419097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadb704-af88-407e-b90b-33374bd7ff08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.421350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 799/1609] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:55.422427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c2ae8-9a86-4fc9-89be-d6119764538c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.432607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c2ae8-9a86-4fc9-89be-d6119764538c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.440813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-dbadb704-af88-407e-b90b-33374bd7ff08 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:55.441998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dbadb704-af88-407e-b90b-33374bd7ff08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:55.459856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:55.492014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f09014-64d2-42d4-b2c4-ed345251369e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.492700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f09014-64d2-42d4-b2c4-ed345251369e admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.492976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f09014-64d2-42d4-b2c4-ed345251369e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.493246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f09014-64d2-42d4-b2c4-ed345251369e admin admin] PATH_INFO: `/v3/projects/80f07909b541427f857887bfe0426de5/users/1b1e2cf23d2f4167aec1221848a5f832/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.501875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 811/1610] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:55.502806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadb704-af88-407e-b90b-33374bd7ff08 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.508993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f09014-64d2-42d4-b2c4-ed345251369e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f07909b541427f857887bfe0426de5, user_id=1b1e2cf23d2f4167aec1221848a5f832, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.512006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f09014-64d2-42d4-b2c4-ed345251369e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.512547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadb704-af88-407e-b90b-33374bd7ff08 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.517469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9d683-84b3-4283-bc8d-3e5aeb94e649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.518268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9d683-84b3-4283-bc8d-3e5aeb94e649 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.518542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9d683-84b3-4283-bc8d-3e5aeb94e649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.518813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9d683-84b3-4283-bc8d-3e5aeb94e649 admin admin] PATH_INFO: `/v3/projects/8d455181e7e046f480437a8ab3620746/users/c4b8d531e382487a938546c009398b2e/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.525737 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 800/1611] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:55 2022] PUT /identity/v3/projects/80f07909b541427f857887bfe0426de5/users/1b1e2cf23d2f4167aec1221848a5f832/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:55.526623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f09014-64d2-42d4-b2c4-ed345251369e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.535250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:55.544654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9d683-84b3-4283-bc8d-3e5aeb94e649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d455181e7e046f480437a8ab3620746, user_id=c4b8d531e382487a938546c009398b2e, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.548212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9d683-84b3-4283-bc8d-3e5aeb94e649 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.562475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 812/1612] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:55 2022] PUT /identity/v3/projects/8d455181e7e046f480437a8ab3620746/users/c4b8d531e382487a938546c009398b2e/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:55.563275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9d683-84b3-4283-bc8d-3e5aeb94e649 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.597117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f09014-64d2-42d4-b2c4-ed345251369e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.625840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9d683-84b3-4283-bc8d-3e5aeb94e649 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.630526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636466e-e22c-4b34-b3b9-63a616b0186e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.631114 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636466e-e22c-4b34-b3b9-63a616b0186e admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.631392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636466e-e22c-4b34-b3b9-63a616b0186e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.631662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636466e-e22c-4b34-b3b9-63a616b0186e admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.632330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636466e-e22c-4b34-b3b9-63a616b0186e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.633929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636466e-e22c-4b34-b3b9-63a616b0186e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.644229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 813/1613] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:55.645233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636466e-e22c-4b34-b3b9-63a616b0186e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.654327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c3262-68e9-40b6-a40b-25ba41d2a78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.655126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c3262-68e9-40b6-a40b-25ba41d2a78c admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.655402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c3262-68e9-40b6-a40b-25ba41d2a78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.655518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636466e-e22c-4b34-b3b9-63a616b0186e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.655671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c3262-68e9-40b6-a40b-25ba41d2a78c admin admin] PATH_INFO: `/v3/projects/9e7aa26bd9e64b30a6c0c3d3824a1dda/users/2e5d00edb03c4b6c91290f1d4bf4c28c/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.660614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85642b10-8d97-4761-9688-46c0189982d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.661210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85642b10-8d97-4761-9688-46c0189982d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.661490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85642b10-8d97-4761-9688-46c0189982d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.661764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85642b10-8d97-4761-9688-46c0189982d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.662386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85642b10-8d97-4761-9688-46c0189982d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.664322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85642b10-8d97-4761-9688-46c0189982d5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.671846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c3262-68e9-40b6-a40b-25ba41d2a78c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7aa26bd9e64b30a6c0c3d3824a1dda, user_id=2e5d00edb03c4b6c91290f1d4bf4c28c, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.675111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c3262-68e9-40b6-a40b-25ba41d2a78c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.689207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 801/1614] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:55 2022] PUT /identity/v3/projects/9e7aa26bd9e64b30a6c0c3d3824a1dda/users/2e5d00edb03c4b6c91290f1d4bf4c28c/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:55.690655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e84d8-1728-4f68-ade0-8a34f287ac4a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.690943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e84d8-1728-4f68-ade0-8a34f287ac4a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.691218 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e84d8-1728-4f68-ade0-8a34f287ac4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.702209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 814/1615] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:55.703291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85642b10-8d97-4761-9688-46c0189982d5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.719356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-697e84d8-1728-4f68-ade0-8a34f287ac4a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:55.720600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-697e84d8-1728-4f68-ade0-8a34f287ac4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:55.765002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85642b10-8d97-4761-9688-46c0189982d5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.769918 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee78e3-13e2-4f7d-b445-4a4a207eda6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.770532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee78e3-13e2-4f7d-b445-4a4a207eda6c admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.770810 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee78e3-13e2-4f7d-b445-4a4a207eda6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.771087 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee78e3-13e2-4f7d-b445-4a4a207eda6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.771730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee78e3-13e2-4f7d-b445-4a4a207eda6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.773384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee78e3-13e2-4f7d-b445-4a4a207eda6c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.787489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 815/1616] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:55.787700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 802/1617] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:55.787700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e4f48-9179-4dd1-8898-f474640ca62a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.787700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e4f48-9179-4dd1-8898-f474640ca62a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.787700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e4f48-9179-4dd1-8898-f474640ca62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.788535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee78e3-13e2-4f7d-b445-4a4a207eda6c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.802687 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:55.816258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-db8e4f48-9179-4dd1-8898-f474640ca62a None None] MFA Rules not processed for user `1b1e2cf23d2f4167aec1221848a5f832`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:55.817474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-db8e4f48-9179-4dd1-8898-f474640ca62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:55.843007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:55.845418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee78e3-13e2-4f7d-b445-4a4a207eda6c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.849657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b61d6-57e7-4ee5-93f4-fcdc8eff673c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.850252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b61d6-57e7-4ee5-93f4-fcdc8eff673c service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.850531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b61d6-57e7-4ee5-93f4-fcdc8eff673c service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.850801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b61d6-57e7-4ee5-93f4-fcdc8eff673c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.859841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b61d6-57e7-4ee5-93f4-fcdc8eff673c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.861933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b61d6-57e7-4ee5-93f4-fcdc8eff673c service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.871364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 816/1618] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:55.872887 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a262e-1f2c-40e0-bb4b-7b27cbcd0749 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.873184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a262e-1f2c-40e0-bb4b-7b27cbcd0749 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.873462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a262e-1f2c-40e0-bb4b-7b27cbcd0749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.885267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 803/1619] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:55.886424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e4f48-9179-4dd1-8898-f474640ca62a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.899944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e4f48-9179-4dd1-8898-f474640ca62a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.906029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8c8a262e-1f2c-40e0-bb4b-7b27cbcd0749 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:55.907281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8c8a262e-1f2c-40e0-bb4b-7b27cbcd0749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:55.912262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e02c0-c2f5-4029-8cfe-38607a1d9e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.913585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e02c0-c2f5-4029-8cfe-38607a1d9e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.914197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e02c0-c2f5-4029-8cfe-38607a1d9e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.914544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e02c0-c2f5-4029-8cfe-38607a1d9e4c admin admin] PATH_INFO: `/v3/projects/bcf7710287bd41e99028493910906a24/users/da187b84e25a4a369aacfba4c5a9aa60/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.917310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 817/1620] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:55.921017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da3f42-6b8a-4d56-9109-1f0c60f59c7f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.921017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da3f42-6b8a-4d56-9109-1f0c60f59c7f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.921017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da3f42-6b8a-4d56-9109-1f0c60f59c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.921343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e02c0-c2f5-4029-8cfe-38607a1d9e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcf7710287bd41e99028493910906a24, user_id=da187b84e25a4a369aacfba4c5a9aa60, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.926331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e02c0-c2f5-4029-8cfe-38607a1d9e4c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.942721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 804/1621] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:55 2022] PUT /identity/v3/projects/bcf7710287bd41e99028493910906a24/users/da187b84e25a4a369aacfba4c5a9aa60/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:55.943874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e02c0-c2f5-4029-8cfe-38607a1d9e4c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.961191 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a5da3f42-6b8a-4d56-9109-1f0c60f59c7f None None] MFA Rules not processed for user `2e5d00edb03c4b6c91290f1d4bf4c28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:55.963332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a5da3f42-6b8a-4d56-9109-1f0c60f59c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:56.021449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e02c0-c2f5-4029-8cfe-38607a1d9e4c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.027335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6e7f7-6128-4f8f-87f9-3698fb1566d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.028008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6e7f7-6128-4f8f-87f9-3698fb1566d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.028311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6e7f7-6128-4f8f-87f9-3698fb1566d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.028614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6e7f7-6128-4f8f-87f9-3698fb1566d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.029450 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6e7f7-6128-4f8f-87f9-3698fb1566d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.031550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6e7f7-6128-4f8f-87f9-3698fb1566d9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.042403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 818/1622] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:56.043596 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da3f42-6b8a-4d56-9109-1f0c60f59c7f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.055799 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da3f42-6b8a-4d56-9109-1f0c60f59c7f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.062929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2861b-f990-4bd5-98d9-3db564c6342a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.064112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2861b-f990-4bd5-98d9-3db564c6342a admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.065142 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2861b-f990-4bd5-98d9-3db564c6342a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.065474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2861b-f990-4bd5-98d9-3db564c6342a admin admin] PATH_INFO: `/v3/projects/fe5205183bbd457a8b84475c5c9582d3/users/9430d6b1327b4fefaf230a89edd8f413/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.076280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 805/1623] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:56.076658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6e7f7-6128-4f8f-87f9-3698fb1566d9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.101159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2861b-f990-4bd5-98d9-3db564c6342a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5205183bbd457a8b84475c5c9582d3, user_id=9430d6b1327b4fefaf230a89edd8f413, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.104432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2861b-f990-4bd5-98d9-3db564c6342a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.126314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 819/1624] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:56 2022] PUT /identity/v3/projects/fe5205183bbd457a8b84475c5c9582d3/users/9430d6b1327b4fefaf230a89edd8f413/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:56.129337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebbcff-75ef-47f8-a97e-08a34d36b474 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.130016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebbcff-75ef-47f8-a97e-08a34d36b474 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.130459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebbcff-75ef-47f8-a97e-08a34d36b474 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.132841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 820/1625] 10.222.0.86 () {54 vars in 962 bytes} [Wed Aug 24 10:02:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:02:56.133965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebbcff-75ef-47f8-a97e-08a34d36b474 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.136551 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1eebbcff-75ef-47f8-a97e-08a34d36b474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:56.156683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6e7f7-6128-4f8f-87f9-3698fb1566d9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.208694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebbcff-75ef-47f8-a97e-08a34d36b474 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.212543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c702f0-da21-4c95-9183-07ade27c2967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.213228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c702f0-da21-4c95-9183-07ade27c2967 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.213395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb80a4-7f57-4b53-bdbf-c6ad2fbbe8a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.213499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c702f0-da21-4c95-9183-07ade27c2967 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.213894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c702f0-da21-4c95-9183-07ade27c2967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.214008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb80a4-7f57-4b53-bdbf-c6ad2fbbe8a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.214191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb80a4-7f57-4b53-bdbf-c6ad2fbbe8a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.214468 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb80a4-7f57-4b53-bdbf-c6ad2fbbe8a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.229132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:56.245381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:56.290274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c702f0-da21-4c95-9183-07ade27c2967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.291916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c702f0-da21-4c95-9183-07ade27c2967 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.304458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb80a4-7f57-4b53-bdbf-c6ad2fbbe8a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.306160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb80a4-7f57-4b53-bdbf-c6ad2fbbe8a5 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.307531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 821/1626] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:56.308782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2bf28-7b9b-4165-b6b2-52e7f3dfc99b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.309162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2bf28-7b9b-4165-b6b2-52e7f3dfc99b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.309656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2bf28-7b9b-4165-b6b2-52e7f3dfc99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.320192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 806/1627] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:56.321806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5fdf4-d00c-48c9-9057-7ece02fb2ffd None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.322250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5fdf4-d00c-48c9-9057-7ece02fb2ffd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.322742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5fdf4-d00c-48c9-9057-7ece02fb2ffd None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.324572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 807/1628] 10.222.0.86 () {54 vars in 962 bytes} [Wed Aug 24 10:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:02:56.329979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5fdf4-d00c-48c9-9057-7ece02fb2ffd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.342846 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d3c2bf28-7b9b-4165-b6b2-52e7f3dfc99b None None] MFA Rules not processed for user `da187b84e25a4a369aacfba4c5a9aa60`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:56.344517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d3c2bf28-7b9b-4165-b6b2-52e7f3dfc99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:56.346053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5fdf4-d00c-48c9-9057-7ece02fb2ffd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.351349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c366e38-ac90-415c-956c-1d4e94f7d319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.351962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c366e38-ac90-415c-956c-1d4e94f7d319 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.352288 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c366e38-ac90-415c-956c-1d4e94f7d319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.352571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c366e38-ac90-415c-956c-1d4e94f7d319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.353589 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c366e38-ac90-415c-956c-1d4e94f7d319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.355925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c366e38-ac90-415c-956c-1d4e94f7d319 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.372361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 808/1629] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:56.373250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c366e38-ac90-415c-956c-1d4e94f7d319 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.388237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c366e38-ac90-415c-956c-1d4e94f7d319 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.396013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bbaff-fcf3-4692-adf5-6cb7ef68e127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.396710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bbaff-fcf3-4692-adf5-6cb7ef68e127 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.397046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bbaff-fcf3-4692-adf5-6cb7ef68e127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.397318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bbaff-fcf3-4692-adf5-6cb7ef68e127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.398147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bbaff-fcf3-4692-adf5-6cb7ef68e127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.400062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bbaff-fcf3-4692-adf5-6cb7ef68e127 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.416445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 809/1630] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:56.418341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd37d24-1dc7-4edd-9afb-9e470b096302 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.419208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd37d24-1dc7-4edd-9afb-9e470b096302 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.419338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd37d24-1dc7-4edd-9afb-9e470b096302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.452725 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 822/1631] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:56.454794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131f09-3653-4a61-850e-e3ceb3c2dcd5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.456921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131f09-3653-4a61-850e-e3ceb3c2dcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.457598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131f09-3653-4a61-850e-e3ceb3c2dcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.479560 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1bd37d24-1dc7-4edd-9afb-9e470b096302 None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:56.481109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1bd37d24-1dc7-4edd-9afb-9e470b096302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:56.488906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-82131f09-3653-4a61-850e-e3ceb3c2dcd5 None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:56.491168 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-82131f09-3653-4a61-850e-e3ceb3c2dcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:56.581877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 810/1632] 10.222.0.86 () {58 vars in 1059 bytes} [Wed Aug 24 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:56.583125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd37d24-1dc7-4edd-9afb-9e470b096302 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.590588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 823/1633] 10.222.0.86 () {58 vars in 1059 bytes} [Wed Aug 24 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:56.591556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131f09-3653-4a61-850e-e3ceb3c2dcd5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.607133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd37d24-1dc7-4edd-9afb-9e470b096302 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.609517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131f09-3653-4a61-850e-e3ceb3c2dcd5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.614790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1662d-f887-427a-9429-7e8f8af5a296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.615614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859ddbd-a518-4f02-8ef2-5d3a0256698e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.615762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1662d-f887-427a-9429-7e8f8af5a296 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.616013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1662d-f887-427a-9429-7e8f8af5a296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.616430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1662d-f887-427a-9429-7e8f8af5a296 admin admin] PATH_INFO: `/v3/projects/2a4021890c6d4c32a03f853b7a2b81ad/users/2d49ff539606427ba3c00a64649dafee/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.616963 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859ddbd-a518-4f02-8ef2-5d3a0256698e admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.617369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859ddbd-a518-4f02-8ef2-5d3a0256698e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.617739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859ddbd-a518-4f02-8ef2-5d3a0256698e admin admin] PATH_INFO: `/v3/projects/fe5205183bbd457a8b84475c5c9582d3/users/9430d6b1327b4fefaf230a89edd8f413/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.620348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859ddbd-a518-4f02-8ef2-5d3a0256698e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5205183bbd457a8b84475c5c9582d3, user_id=9430d6b1327b4fefaf230a89edd8f413, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.625432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859ddbd-a518-4f02-8ef2-5d3a0256698e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.639129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1662d-f887-427a-9429-7e8f8af5a296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4021890c6d4c32a03f853b7a2b81ad, user_id=2d49ff539606427ba3c00a64649dafee, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.641107 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 824/1634] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:56 2022] PUT /identity/v3/projects/fe5205183bbd457a8b84475c5c9582d3/users/9430d6b1327b4fefaf230a89edd8f413/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:56.641960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859ddbd-a518-4f02-8ef2-5d3a0256698e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.642824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1662d-f887-427a-9429-7e8f8af5a296 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.665222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 811/1635] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:56 2022] PUT /identity/v3/projects/2a4021890c6d4c32a03f853b7a2b81ad/users/2d49ff539606427ba3c00a64649dafee/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:56.666573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1662d-f887-427a-9429-7e8f8af5a296 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.702374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:56.734675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:56.745836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859ddbd-a518-4f02-8ef2-5d3a0256698e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.770003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1662d-f887-427a-9429-7e8f8af5a296 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.787351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64971a3-cc6c-4d9d-bda3-342ab6e8b861 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.788008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64971a3-cc6c-4d9d-bda3-342ab6e8b861 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.788360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64971a3-cc6c-4d9d-bda3-342ab6e8b861 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.788699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64971a3-cc6c-4d9d-bda3-342ab6e8b861 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.846847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec78804-c16f-452e-b056-dd5ba2035933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.847500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec78804-c16f-452e-b056-dd5ba2035933 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.847801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec78804-c16f-452e-b056-dd5ba2035933 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.848279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec78804-c16f-452e-b056-dd5ba2035933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.959918 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec78804-c16f-452e-b056-dd5ba2035933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.961939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec78804-c16f-452e-b056-dd5ba2035933 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.968582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64971a3-cc6c-4d9d-bda3-342ab6e8b861 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.972435 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64971a3-cc6c-4d9d-bda3-342ab6e8b861 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.975501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 825/1636] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:56.976831 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec78804-c16f-452e-b056-dd5ba2035933 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.993774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec78804-c16f-452e-b056-dd5ba2035933 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:57.001043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5af94-af85-40c5-98a7-7afc58df5e15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:57.001840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5af94-af85-40c5-98a7-7afc58df5e15 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.002174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5af94-af85-40c5-98a7-7afc58df5e15 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.002775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5af94-af85-40c5-98a7-7afc58df5e15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.015211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 812/1637] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:57.034597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aab13e-cb31-4ced-98dc-57bb161e7354 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.036767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aab13e-cb31-4ced-98dc-57bb161e7354 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.037338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aab13e-cb31-4ced-98dc-57bb161e7354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.071864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5af94-af85-40c5-98a7-7afc58df5e15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:57.074419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5af94-af85-40c5-98a7-7afc58df5e15 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:57.094138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 826/1638] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:57.096739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5af94-af85-40c5-98a7-7afc58df5e15 service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:57.144540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f8aab13e-cb31-4ced-98dc-57bb161e7354 None None] MFA Rules not processed for user `9430d6b1327b4fefaf230a89edd8f413`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:57.146485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:57.148544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f8aab13e-cb31-4ced-98dc-57bb161e7354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:57.187036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5af94-af85-40c5-98a7-7afc58df5e15 service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:57.200303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee95dde-4fcd-4696-bf6a-3403b3f070b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:57.200942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee95dde-4fcd-4696-bf6a-3403b3f070b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.201368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee95dde-4fcd-4696-bf6a-3403b3f070b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.201679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee95dde-4fcd-4696-bf6a-3403b3f070b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.202364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee95dde-4fcd-4696-bf6a-3403b3f070b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:57.204087 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee95dde-4fcd-4696-bf6a-3403b3f070b0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:57.230129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 827/1639] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:57.231208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee95dde-4fcd-4696-bf6a-3403b3f070b0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:57.246209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee95dde-4fcd-4696-bf6a-3403b3f070b0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:57.251112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfc47a-81bc-434f-9f61-d16338fc3fc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:57.252547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfc47a-81bc-434f-9f61-d16338fc3fc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.252837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfc47a-81bc-434f-9f61-d16338fc3fc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.253112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfc47a-81bc-434f-9f61-d16338fc3fc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.327223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dfc47a-81bc-434f-9f61-d16338fc3fc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:57.328092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 813/1640] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:57.329400 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dfc47a-81bc-434f-9f61-d16338fc3fc1 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:57.340127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aab13e-cb31-4ced-98dc-57bb161e7354 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:57.347606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 828/1641] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:57.348687 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfc47a-81bc-434f-9f61-d16338fc3fc1 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:57.363367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfc47a-81bc-434f-9f61-d16338fc3fc1 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:57.368565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18a60c-a906-4c00-87a4-091e739c9edc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:57.369271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18a60c-a906-4c00-87a4-091e739c9edc service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.369623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18a60c-a906-4c00-87a4-091e739c9edc service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.369898 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18a60c-a906-4c00-87a4-091e739c9edc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.388658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aab13e-cb31-4ced-98dc-57bb161e7354 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:57.405266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10789439-3086-488a-ae62-d529d67f75fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:57.406054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10789439-3086-488a-ae62-d529d67f75fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.407540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10789439-3086-488a-ae62-d529d67f75fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.407917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10789439-3086-488a-ae62-d529d67f75fb admin admin] PATH_INFO: `/v3/projects/2a4021890c6d4c32a03f853b7a2b81ad/users/2d49ff539606427ba3c00a64649dafee/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.417506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10789439-3086-488a-ae62-d529d67f75fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4021890c6d4c32a03f853b7a2b81ad, user_id=2d49ff539606427ba3c00a64649dafee, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:57.424610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10789439-3086-488a-ae62-d529d67f75fb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:57.455725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 814/1642] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:57 2022] PUT /identity/v3/projects/2a4021890c6d4c32a03f853b7a2b81ad/users/2d49ff539606427ba3c00a64649dafee/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:57.465961 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995a577-bc2e-4e87-be16-d48b3d2fbcf3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.466239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995a577-bc2e-4e87-be16-d48b3d2fbcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.466538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995a577-bc2e-4e87-be16-d48b3d2fbcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.514827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18a60c-a906-4c00-87a4-091e739c9edc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:57.516330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18a60c-a906-4c00-87a4-091e739c9edc service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:57.536155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3995a577-bc2e-4e87-be16-d48b3d2fbcf3 None None] MFA Rules not processed for user `2d49ff539606427ba3c00a64649dafee`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:57.539341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3995a577-bc2e-4e87-be16-d48b3d2fbcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:57.541034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 829/1643] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:57.594806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:57.636131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 815/1644] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:02:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:57.673962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632a112-5fb2-4850-afda-a7c94f4fd195 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.674251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632a112-5fb2-4850-afda-a7c94f4fd195 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.674532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632a112-5fb2-4850-afda-a7c94f4fd195 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.676415 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 830/1645] 10.222.0.86 () {54 vars in 912 bytes} [Wed Aug 24 10:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:02:57.682025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a3a9e-9f3b-4084-bd66-caaea774ca87 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.682322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a3a9e-9f3b-4084-bd66-caaea774ca87 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.682599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a3a9e-9f3b-4084-bd66-caaea774ca87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.684456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24232a-0b5e-4723-aef1-aaa825919dd6 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.684738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24232a-0b5e-4723-aef1-aaa825919dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.685010 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24232a-0b5e-4723-aef1-aaa825919dd6 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.686377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 831/1646] 10.222.0.86 () {54 vars in 912 bytes} [Wed Aug 24 10:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:02:57.693421 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfca448-0f69-4357-a9cc-b85e3421b207 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.694133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfca448-0f69-4357-a9cc-b85e3421b207 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.694842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfca448-0f69-4357-a9cc-b85e3421b207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.725934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7bfca448-0f69-4357-a9cc-b85e3421b207 None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:57.727343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7bfca448-0f69-4357-a9cc-b85e3421b207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:57.736001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b08a3a9e-9f3b-4084-bd66-caaea774ca87 None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:57.744224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b08a3a9e-9f3b-4084-bd66-caaea774ca87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:57.855973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:57.905701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 832/1647] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:02:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:57.908930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfca448-0f69-4357-a9cc-b85e3421b207 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:58.000741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 816/1648] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:02:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:58.045446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfca448-0f69-4357-a9cc-b85e3421b207 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:58.054285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a8811-206a-4e4c-9d5b-d80d22717168 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:58.055302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a8811-206a-4e4c-9d5b-d80d22717168 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:58.058453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a8811-206a-4e4c-9d5b-d80d22717168 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:58.058875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a8811-206a-4e4c-9d5b-d80d22717168 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:58.082070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a8811-206a-4e4c-9d5b-d80d22717168 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:58.084149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a8811-206a-4e4c-9d5b-d80d22717168 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:58.110958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 833/1649] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:58.283514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a3a9e-9f3b-4084-bd66-caaea774ca87 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:58.294997 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a3a9e-9f3b-4084-bd66-caaea774ca87 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:58.305307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d8b54-a492-4e9c-a5ba-fd20313eb877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:58.305957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8b54-a492-4e9c-a5ba-fd20313eb877 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:58.306692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8b54-a492-4e9c-a5ba-fd20313eb877 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:58.306692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8b54-a492-4e9c-a5ba-fd20313eb877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:58.340971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d8b54-a492-4e9c-a5ba-fd20313eb877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:58.342681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d8b54-a492-4e9c-a5ba-fd20313eb877 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:58.370149 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 817/1650] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:58.502574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a8811-206a-4e4c-9d5b-d80d22717168 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:58.514152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a8811-206a-4e4c-9d5b-d80d22717168 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:58.523206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595dd90-f45a-4b7d-9d96-40187fcbadaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:58.525033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595dd90-f45a-4b7d-9d96-40187fcbadaa service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:58.525556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595dd90-f45a-4b7d-9d96-40187fcbadaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:58.526045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595dd90-f45a-4b7d-9d96-40187fcbadaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:58.555263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595dd90-f45a-4b7d-9d96-40187fcbadaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:58.556938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595dd90-f45a-4b7d-9d96-40187fcbadaa service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:58.586745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 834/1651] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:00.606327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d8b54-a492-4e9c-a5ba-fd20313eb877 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:00.873415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d8b54-a492-4e9c-a5ba-fd20313eb877 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:00.948431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:00.948431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:00.948431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:00.948431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] PATH_INFO: `/v3/users/cb051667aa484eafa4fa6b66c15d649e` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:00.952607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb051667aa484eafa4fa6b66c15d649e)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:00.952607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:01.124272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb051667aa484eafa4fa6b66c15d649e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:01.125016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb051667aa484eafa4fa6b66c15d649e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:01.125653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb051667aa484eafa4fa6b66c15d649e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:01.151663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb051667aa484eafa4fa6b66c15d649e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:01.160234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:01.208110 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb051667aa484eafa4fa6b66c15d649e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:01.294661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 818/1652] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:00 2022] DELETE /identity/v3/users/cb051667aa484eafa4fa6b66c15d649e => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:01.300636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1251f-de40-472f-b093-314c75bcf89b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:01.302693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1251f-de40-472f-b093-314c75bcf89b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:01.303141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1251f-de40-472f-b093-314c75bcf89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:01.362743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8ac1251f-de40-472f-b093-314c75bcf89b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:01.365554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8ac1251f-de40-472f-b093-314c75bcf89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:01.562242 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 835/1653] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:01.581327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:01.582876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:01.750058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae0bf-caa1-4771-955f-4fdaae33a616 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:01.792767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2cd7c-1df0-4ee7-95cd-7f3897a82fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:01.793398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2cd7c-1df0-4ee7-95cd-7f3897a82fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:01.793693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2cd7c-1df0-4ee7-95cd-7f3897a82fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:01.793944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2cd7c-1df0-4ee7-95cd-7f3897a82fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:01.818129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2cd7c-1df0-4ee7-95cd-7f3897a82fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:01.827013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2cd7c-1df0-4ee7-95cd-7f3897a82fe2 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:01.873473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 819/1654] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:03.192310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1251f-de40-472f-b093-314c75bcf89b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:03.193892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8ac1251f-de40-472f-b093-314c75bcf89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:03.212468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1251f-de40-472f-b093-314c75bcf89b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:03.226262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:03.226923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:03.227204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:03.227479 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] PATH_INFO: `/v3/projects/d269b3486e3b4375800858b532f78f35` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:03.237277 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d269b3486e3b4375800858b532f78f35)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:03.239383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:03.333827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd269b3486e3b4375800858b532f78f35', 'request_id': 'req-780173fd-d12b-484b-b275-c69ccc71c1b1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.359485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd269b3486e3b4375800858b532f78f35', 'request_id': 'req-780173fd-d12b-484b-b275-c69ccc71c1b1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.384210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd269b3486e3b4375800858b532f78f35', 'request_id': 'req-780173fd-d12b-484b-b275-c69ccc71c1b1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.406434 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] The token cache is being invalidate because project d269b3486e3b4375800858b532f78f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:03.406729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.409548 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.415758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 836/1655] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:03 2022] DELETE /identity/v3/projects/d269b3486e3b4375800858b532f78f35 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:03.419959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2cd7c-1df0-4ee7-95cd-7f3897a82fe2 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:03.432336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2cd7c-1df0-4ee7-95cd-7f3897a82fe2 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:03.605917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:03.606706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:03.606987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:03.607257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] PATH_INFO: `/v3/users/2d49ff539606427ba3c00a64649dafee` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:03.610059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d49ff539606427ba3c00a64649dafee)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:03.612435 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:03.721934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d49ff539606427ba3c00a64649dafee'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.724813 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d49ff539606427ba3c00a64649dafee'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.728286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d49ff539606427ba3c00a64649dafee'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.745787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d49ff539606427ba3c00a64649dafee'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.759777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d49ff539606427ba3c00a64649dafee'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.807764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:03.822899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 820/1656] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:03 2022] DELETE /identity/v3/users/2d49ff539606427ba3c00a64649dafee => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:04.451841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:04.453314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:04.487005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:04.591422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780173fd-d12b-484b-b275-c69ccc71c1b1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:04.694396 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe27ca6-3c84-438d-a33f-1d67177ea43e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:04.695718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe27ca6-3c84-438d-a33f-1d67177ea43e service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:04.696191 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe27ca6-3c84-438d-a33f-1d67177ea43e service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:04.696643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe27ca6-3c84-438d-a33f-1d67177ea43e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:04.699361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-afe27ca6-3c84-438d-a33f-1d67177ea43e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:04.878503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe27ca6-3c84-438d-a33f-1d67177ea43e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:04.879987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe27ca6-3c84-438d-a33f-1d67177ea43e service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:04.906661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 837/1657] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:05.249857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:05.251354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:05.498494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63f6be-de2d-4e95-b16f-7adaa7481741 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:05.519253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:05.519915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.520237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.520510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] PATH_INFO: `/v3/projects/2a4021890c6d4c32a03f853b7a2b81ad` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.525489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4021890c6d4c32a03f853b7a2b81ad)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:05.526751 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:03:05.526751 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 24 10:03:05.526751 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: Aug 24 10:03:05.527980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:05.704240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4021890c6d4c32a03f853b7a2b81ad', 'request_id': 'req-148ad2f1-5284-4d96-857c-0ac9c4f090fc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:05.724312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4021890c6d4c32a03f853b7a2b81ad', 'request_id': 'req-148ad2f1-5284-4d96-857c-0ac9c4f090fc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:05.732060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4021890c6d4c32a03f853b7a2b81ad', 'request_id': 'req-148ad2f1-5284-4d96-857c-0ac9c4f090fc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:05.763397 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] The token cache is being invalidate because project 2a4021890c6d4c32a03f853b7a2b81ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:05.764089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:05.765103 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-148ad2f1-5284-4d96-857c-0ac9c4f090fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:05.792624 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 821/1658] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:05 2022] DELETE /identity/v3/projects/2a4021890c6d4c32a03f853b7a2b81ad => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:05.797974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bdb78-9424-470b-93f2-d537aabe104b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.798425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bdb78-9424-470b-93f2-d537aabe104b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.798617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bdb78-9424-470b-93f2-d537aabe104b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.859872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-9e2bdb78-9424-470b-93f2-d537aabe104b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:05.860530 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9e2bdb78-9424-470b-93f2-d537aabe104b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:05.925520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b35d6-020d-4cce-a313-a94bb9f4a8d5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.925844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b35d6-020d-4cce-a313-a94bb9f4a8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.926133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b35d6-020d-4cce-a313-a94bb9f4a8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.963316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 838/1659] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:05.973585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23724db7-da43-4c30-9507-5aaf85128cb2 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.974150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23724db7-da43-4c30-9507-5aaf85128cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.974725 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23724db7-da43-4c30-9507-5aaf85128cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.983832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1a8b35d6-020d-4cce-a313-a94bb9f4a8d5 None None] MFA Rules not processed for user `df9b402589fa4020bc241a4b9c06dd91`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:05.985294 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1a8b35d6-020d-4cce-a313-a94bb9f4a8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:06.000793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-23724db7-da43-4c30-9507-5aaf85128cb2 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:06.003682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-23724db7-da43-4c30-9507-5aaf85128cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:06.027919 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 839/1660] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:06.028870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23724db7-da43-4c30-9507-5aaf85128cb2 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:06.039025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23724db7-da43-4c30-9507-5aaf85128cb2 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:06.048311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2132ce-f6ea-4cf8-b9ec-771d34b9a8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:06.048941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2132ce-f6ea-4cf8-b9ec-771d34b9a8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:06.049219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2132ce-f6ea-4cf8-b9ec-771d34b9a8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:06.049519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2132ce-f6ea-4cf8-b9ec-771d34b9a8b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:06.050178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2132ce-f6ea-4cf8-b9ec-771d34b9a8b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:06.050483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2132ce-f6ea-4cf8-b9ec-771d34b9a8b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:06.052156 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2132ce-f6ea-4cf8-b9ec-771d34b9a8b5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:06.061435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:06.078386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 840/1661] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:06.080153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2132ce-f6ea-4cf8-b9ec-771d34b9a8b5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:06.090365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2132ce-f6ea-4cf8-b9ec-771d34b9a8b5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:06.096969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf09bb-e647-48bd-8dfa-24a765bfb02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:06.097656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf09bb-e647-48bd-8dfa-24a765bfb02d admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:06.097941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf09bb-e647-48bd-8dfa-24a765bfb02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:06.098362 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf09bb-e647-48bd-8dfa-24a765bfb02d admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:06.099055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf09bb-e647-48bd-8dfa-24a765bfb02d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:06.101227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf09bb-e647-48bd-8dfa-24a765bfb02d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:06.148220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 841/1662] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:06 2022] POST /identity/v3/projects => generated 357 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:06.150719 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf09bb-e647-48bd-8dfa-24a765bfb02d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:06.179908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 822/1663] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:06.259685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b35d6-020d-4cce-a313-a94bb9f4a8d5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:06.264777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1a8b35d6-020d-4cce-a313-a94bb9f4a8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:06.305716 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:06.354849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf09bb-e647-48bd-8dfa-24a765bfb02d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:06.361658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190c79d-3412-404e-ada1-36c9ac973969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:06.362457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190c79d-3412-404e-ada1-36c9ac973969 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:06.362750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190c79d-3412-404e-ada1-36c9ac973969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:06.363019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190c79d-3412-404e-ada1-36c9ac973969 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:06.363666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190c79d-3412-404e-ada1-36c9ac973969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:06.365900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190c79d-3412-404e-ada1-36c9ac973969 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:06.399144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 842/1664] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:06 2022] POST /identity/v3/users => generated 279 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:06.401030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190c79d-3412-404e-ada1-36c9ac973969 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:06.403710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b35d6-020d-4cce-a313-a94bb9f4a8d5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:06.418781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190c79d-3412-404e-ada1-36c9ac973969 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:06.426255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc541e7-034f-4b52-a3ae-7baeeebcda00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:06.427044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc541e7-034f-4b52-a3ae-7baeeebcda00 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:06.427350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc541e7-034f-4b52-a3ae-7baeeebcda00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:06.428430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc541e7-034f-4b52-a3ae-7baeeebcda00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:06.428863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc541e7-034f-4b52-a3ae-7baeeebcda00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:06.430681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc541e7-034f-4b52-a3ae-7baeeebcda00 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:06.453131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 843/1665] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:06.457150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc541e7-034f-4b52-a3ae-7baeeebcda00 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:06.472380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc541e7-034f-4b52-a3ae-7baeeebcda00 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:06.481684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c4f33-d74c-45a2-aced-d92c215740ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:06.482313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c4f33-d74c-45a2-aced-d92c215740ae admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:06.482667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c4f33-d74c-45a2-aced-d92c215740ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:06.482949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c4f33-d74c-45a2-aced-d92c215740ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:06.483867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c4f33-d74c-45a2-aced-d92c215740ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:06.485746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c4f33-d74c-45a2-aced-d92c215740ae admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:06.510113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 844/1666] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:06.512203 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c4f33-d74c-45a2-aced-d92c215740ae admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:06.541868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c4f33-d74c-45a2-aced-d92c215740ae admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:06.548862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd7995-b1db-418f-a51f-8821f3dd8ee7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:06.549497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd7995-b1db-418f-a51f-8821f3dd8ee7 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:06.550356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd7995-b1db-418f-a51f-8821f3dd8ee7 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:06.550801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1613b-a3e9-4293-b280-7a56565a9303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:06.552072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1613b-a3e9-4293-b280-7a56565a9303 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:06.552492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1613b-a3e9-4293-b280-7a56565a9303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:06.552715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd7995-b1db-418f-a51f-8821f3dd8ee7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:06.552906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1613b-a3e9-4293-b280-7a56565a9303 admin admin] PATH_INFO: `/v3/projects/53f4489793504fba95d11f1e3b930842/users/ff39a263247a45f8a2fe175aa63a9fd2/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:06.580778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1613b-a3e9-4293-b280-7a56565a9303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f4489793504fba95d11f1e3b930842, user_id=ff39a263247a45f8a2fe175aa63a9fd2, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:06.588543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1613b-a3e9-4293-b280-7a56565a9303 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:06.626216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 845/1667] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:06 2022] PUT /identity/v3/projects/53f4489793504fba95d11f1e3b930842/users/ff39a263247a45f8a2fe175aa63a9fd2/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:06.628304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dc34c-54f7-431f-b049-ac0fbb8bdae2 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:06.628636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dc34c-54f7-431f-b049-ac0fbb8bdae2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:06.629160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dc34c-54f7-431f-b049-ac0fbb8bdae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:06.674794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e47dc34c-54f7-431f-b049-ac0fbb8bdae2 None None] MFA Rules not processed for user `ff39a263247a45f8a2fe175aa63a9fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:06.678175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e47dc34c-54f7-431f-b049-ac0fbb8bdae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:06.692782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd7995-b1db-418f-a51f-8821f3dd8ee7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:06.694325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd7995-b1db-418f-a51f-8821f3dd8ee7 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:06.722140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 823/1668] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:06.806707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 846/1669] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:06.808131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd7995-b1db-418f-a51f-8821f3dd8ee7 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:06.828661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd7995-b1db-418f-a51f-8821f3dd8ee7 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:06.951996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:06.972929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9fa1c-24d9-4d0c-a6e7-1c39bbd31c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:06.973479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9fa1c-24d9-4d0c-a6e7-1c39bbd31c25 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:06.973757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9fa1c-24d9-4d0c-a6e7-1c39bbd31c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:06.974028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9fa1c-24d9-4d0c-a6e7-1c39bbd31c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:06.975139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9fa1c-24d9-4d0c-a6e7-1c39bbd31c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:06.976608 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9fa1c-24d9-4d0c-a6e7-1c39bbd31c25 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:07.045388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 824/1670] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:06 2022] POST /identity/v3/projects => generated 357 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:07.049602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dc34c-54f7-431f-b049-ac0fbb8bdae2 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:07.072401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dc34c-54f7-431f-b049-ac0fbb8bdae2 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:07.123915 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:07.230585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a2bbc-c854-4790-b1b9-0d817e73de69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:07.232658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a2bbc-c854-4790-b1b9-0d817e73de69 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:07.233454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a2bbc-c854-4790-b1b9-0d817e73de69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:07.234305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a2bbc-c854-4790-b1b9-0d817e73de69 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:07.241354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a2bbc-c854-4790-b1b9-0d817e73de69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:07.245337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a2bbc-c854-4790-b1b9-0d817e73de69 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:07.330905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 847/1671] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:07 2022] POST /identity/v3/users => generated 285 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:07.334104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9fa1c-24d9-4d0c-a6e7-1c39bbd31c25 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:07.349831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9fa1c-24d9-4d0c-a6e7-1c39bbd31c25 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:07.374971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103b8aa-1c06-4b40-8b2c-6544702c9e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:07.375230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103b8aa-1c06-4b40-8b2c-6544702c9e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:07.375673 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103b8aa-1c06-4b40-8b2c-6544702c9e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:07.377092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103b8aa-1c06-4b40-8b2c-6544702c9e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:07.378056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103b8aa-1c06-4b40-8b2c-6544702c9e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:07.379924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103b8aa-1c06-4b40-8b2c-6544702c9e2b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:07.412031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 825/1672] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:07.416139 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a2bbc-c854-4790-b1b9-0d817e73de69 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:07.434560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a2bbc-c854-4790-b1b9-0d817e73de69 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:07.450018 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71824c72-8f7c-4e9e-a76b-861d1d8b38ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:07.450708 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71824c72-8f7c-4e9e-a76b-861d1d8b38ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:07.450985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71824c72-8f7c-4e9e-a76b-861d1d8b38ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:07.451258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71824c72-8f7c-4e9e-a76b-861d1d8b38ca admin admin] PATH_INFO: `/v3/projects/d7e9a17f65ca488e8939acad5497831a/users/ffe0758ed24d4832a544df58efc3fa35/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:07.514164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71824c72-8f7c-4e9e-a76b-861d1d8b38ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e9a17f65ca488e8939acad5497831a, user_id=ffe0758ed24d4832a544df58efc3fa35, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:07.520475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71824c72-8f7c-4e9e-a76b-861d1d8b38ca admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:07.557370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 848/1673] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:07 2022] PUT /identity/v3/projects/d7e9a17f65ca488e8939acad5497831a/users/ffe0758ed24d4832a544df58efc3fa35/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:07.561266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103b8aa-1c06-4b40-8b2c-6544702c9e2b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:07.578467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103b8aa-1c06-4b40-8b2c-6544702c9e2b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:07.737236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872c545-0558-4c4c-b354-e7c4c555286a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:07.737868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872c545-0558-4c4c-b354-e7c4c555286a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:07.738148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872c545-0558-4c4c-b354-e7c4c555286a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:07.738418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872c545-0558-4c4c-b354-e7c4c555286a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:07.739210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872c545-0558-4c4c-b354-e7c4c555286a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:07.742782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872c545-0558-4c4c-b354-e7c4c555286a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:07.797015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 826/1674] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:07.801588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71824c72-8f7c-4e9e-a76b-861d1d8b38ca admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:07.821769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71824c72-8f7c-4e9e-a76b-861d1d8b38ca admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:07.838785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7b14f-9cce-489e-9245-fceba09c6c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:07.841291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7b14f-9cce-489e-9245-fceba09c6c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:07.841776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7b14f-9cce-489e-9245-fceba09c6c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:07.842358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7b14f-9cce-489e-9245-fceba09c6c17 admin admin] PATH_INFO: `/v3/projects/d7e9a17f65ca488e8939acad5497831a/users/ffe0758ed24d4832a544df58efc3fa35/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:07.847006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7b14f-9cce-489e-9245-fceba09c6c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e9a17f65ca488e8939acad5497831a, user_id=ffe0758ed24d4832a544df58efc3fa35, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:07.851910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7b14f-9cce-489e-9245-fceba09c6c17 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:07.886065 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 849/1675] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:07 2022] PUT /identity/v3/projects/d7e9a17f65ca488e8939acad5497831a/users/ffe0758ed24d4832a544df58efc3fa35/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:07.889259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb18383-68c1-4f50-8982-4b1c86abbd72 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:07.889606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb18383-68c1-4f50-8982-4b1c86abbd72 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:07.889898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb18383-68c1-4f50-8982-4b1c86abbd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:08.003738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-dbb18383-68c1-4f50-8982-4b1c86abbd72 None None] MFA Rules not processed for user `ffe0758ed24d4832a544df58efc3fa35`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:08.005036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dbb18383-68c1-4f50-8982-4b1c86abbd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:08.208267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 827/1676] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:08.228614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682aee8-8418-4d30-a658-0f9309134b7f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:08.228910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682aee8-8418-4d30-a658-0f9309134b7f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:08.229255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682aee8-8418-4d30-a658-0f9309134b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:08.237079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb18383-68c1-4f50-8982-4b1c86abbd72 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:08.238478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dbb18383-68c1-4f50-8982-4b1c86abbd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:08.367043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8682aee8-8418-4d30-a658-0f9309134b7f None None] MFA Rules not processed for user `46c01f8b99e24be5b6c2d73b9b1bd9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:08.368930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8682aee8-8418-4d30-a658-0f9309134b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:08.375056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:08.388021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb18383-68c1-4f50-8982-4b1c86abbd72 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:08.422868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05daffd9-7610-4d51-9948-542e31887573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:08.422868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daffd9-7610-4d51-9948-542e31887573 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:08.422868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daffd9-7610-4d51-9948-542e31887573 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:08.422868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daffd9-7610-4d51-9948-542e31887573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:08.437685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05daffd9-7610-4d51-9948-542e31887573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:08.439999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05daffd9-7610-4d51-9948-542e31887573 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:08.459571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 828/1677] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:08 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:08.646566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 850/1678] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:08.661686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a508ca7-3874-47a6-a683-fe4c46749bf4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:08.661984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a508ca7-3874-47a6-a683-fe4c46749bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:08.662252 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a508ca7-3874-47a6-a683-fe4c46749bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:08.694167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3a508ca7-3874-47a6-a683-fe4c46749bf4 None None] MFA Rules not processed for user `46c01f8b99e24be5b6c2d73b9b1bd9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:08.695652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3a508ca7-3874-47a6-a683-fe4c46749bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:08.731520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 829/1679] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:08.741758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682aee8-8418-4d30-a658-0f9309134b7f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:08.758193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a508ca7-3874-47a6-a683-fe4c46749bf4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:08.760538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3a508ca7-3874-47a6-a683-fe4c46749bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:08.802433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a508ca7-3874-47a6-a683-fe4c46749bf4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:08.808754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d09a5d-55b4-452e-90c0-83ba1fa41ea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:08.809607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d09a5d-55b4-452e-90c0-83ba1fa41ea0 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:08.810022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d09a5d-55b4-452e-90c0-83ba1fa41ea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:08.810437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d09a5d-55b4-452e-90c0-83ba1fa41ea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:08.873206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:08.939486 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682aee8-8418-4d30-a658-0f9309134b7f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:08.951858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f8c59-f7b7-4e86-9851-3b3021ebcb8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:08.952538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f8c59-f7b7-4e86-9851-3b3021ebcb8f service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:08.952813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f8c59-f7b7-4e86-9851-3b3021ebcb8f service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:08.953082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f8c59-f7b7-4e86-9851-3b3021ebcb8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:08.974846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d09a5d-55b4-452e-90c0-83ba1fa41ea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:08.978348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d09a5d-55b4-452e-90c0-83ba1fa41ea0 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:08.989763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f8c59-f7b7-4e86-9851-3b3021ebcb8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:08.991376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f8c59-f7b7-4e86-9851-3b3021ebcb8f service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:09.014351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 830/1680] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:09.038983 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 851/1681] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:09.828428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf52ce-b5ba-4658-8c23-d8261e4433e9 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:09.829344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf52ce-b5ba-4658-8c23-d8261e4433e9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:09.830476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf52ce-b5ba-4658-8c23-d8261e4433e9 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:09.832501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 831/1682] 10.222.0.86 () {54 vars in 912 bytes} [Wed Aug 24 10:03:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:03:09.839643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde73e94-2e45-43c1-9253-b0be34a7c1e4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:09.840489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde73e94-2e45-43c1-9253-b0be34a7c1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:09.842675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde73e94-2e45-43c1-9253-b0be34a7c1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:09.894423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97575f92-8f94-4b21-8b9f-5a9a2f03fc46 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:09.894842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97575f92-8f94-4b21-8b9f-5a9a2f03fc46 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:09.894965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97575f92-8f94-4b21-8b9f-5a9a2f03fc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:09.919712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fde73e94-2e45-43c1-9253-b0be34a7c1e4 None None] MFA Rules not processed for user `27cc091f3b094cb1a1e2650a3fbd7671`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:09.930621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fde73e94-2e45-43c1-9253-b0be34a7c1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:09.952321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-97575f92-8f94-4b21-8b9f-5a9a2f03fc46 None None] MFA Rules not processed for user `df9b402589fa4020bc241a4b9c06dd91`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:09.953477 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-97575f92-8f94-4b21-8b9f-5a9a2f03fc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:10.035875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 852/1683] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:03:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:10.042513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64361d-21a1-4785-92ef-ac44554514bd None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:10.042513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64361d-21a1-4785-92ef-ac44554514bd None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:10.042664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64361d-21a1-4785-92ef-ac44554514bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:10.062193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 832/1684] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:10.063224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97575f92-8f94-4b21-8b9f-5a9a2f03fc46 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:10.071498 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-97575f92-8f94-4b21-8b9f-5a9a2f03fc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:10.082042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97575f92-8f94-4b21-8b9f-5a9a2f03fc46 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:10.086659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e408aa-0aeb-4d96-aacf-9f7d9e3cf7a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:10.087844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e408aa-0aeb-4d96-aacf-9f7d9e3cf7a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:10.087844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e408aa-0aeb-4d96-aacf-9f7d9e3cf7a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:10.088226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e408aa-0aeb-4d96-aacf-9f7d9e3cf7a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:10.093425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ea64361d-21a1-4785-92ef-ac44554514bd None None] MFA Rules not processed for user `c4b8d531e382487a938546c009398b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:10.094980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ea64361d-21a1-4785-92ef-ac44554514bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:10.100870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e408aa-0aeb-4d96-aacf-9f7d9e3cf7a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:10.103070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e408aa-0aeb-4d96-aacf-9f7d9e3cf7a1 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:10.113425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 833/1685] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:10.185733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 853/1686] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:10.196743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd0bb8-f36f-4543-9549-554fd69bd450 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:10.196945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd0bb8-f36f-4543-9549-554fd69bd450 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:10.197130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd0bb8-f36f-4543-9549-554fd69bd450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:10.219012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-77fd0bb8-f36f-4543-9549-554fd69bd450 None None] MFA Rules not processed for user `c4b8d531e382487a938546c009398b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:10.220372 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-77fd0bb8-f36f-4543-9549-554fd69bd450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:10.241310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 834/1687] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:10.630802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64361d-21a1-4785-92ef-ac44554514bd None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:10.653598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64361d-21a1-4785-92ef-ac44554514bd None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:10.676574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0bc35-ee35-4618-8037-5bc5dff08f47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:10.680629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0bc35-ee35-4618-8037-5bc5dff08f47 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:10.692850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0bc35-ee35-4618-8037-5bc5dff08f47 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:10.693476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0bc35-ee35-4618-8037-5bc5dff08f47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:10.728628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0bc35-ee35-4618-8037-5bc5dff08f47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:10.747126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0bc35-ee35-4618-8037-5bc5dff08f47 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:10.776467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 854/1688] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:10.789210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c7f35-68ed-4363-aea3-a4f3aa629161 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:10.789790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c7f35-68ed-4363-aea3-a4f3aa629161 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:10.790422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c7f35-68ed-4363-aea3-a4f3aa629161 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:10.793257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 835/1689] 10.222.0.86 () {54 vars in 961 bytes} [Wed Aug 24 10:03:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:03:10.823192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f3204-23e9-415e-bdc2-8c6f4e1799a3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:10.824467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f3204-23e9-415e-bdc2-8c6f4e1799a3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:10.827227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f3204-23e9-415e-bdc2-8c6f4e1799a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:10.861787 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-696f3204-23e9-415e-bdc2-8c6f4e1799a3 None None] MFA Rules not processed for user `ef8d30e87c624d6180656d998afcebe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:10.863947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-696f3204-23e9-415e-bdc2-8c6f4e1799a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:10.972606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:11.020495 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65658f43-9c8f-4e01-85a4-b214d7ce64db None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:11.021726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65658f43-9c8f-4e01-85a4-b214d7ce64db None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:11.022328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65658f43-9c8f-4e01-85a4-b214d7ce64db None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:11.024656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 836/1690] 10.222.0.86 () {54 vars in 912 bytes} [Wed Aug 24 10:03:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:03:11.026602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 855/1691] 10.222.0.86 () {58 vars in 1058 bytes} [Wed Aug 24 10:03:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:11.033263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5787f1e-b3d3-4475-b59e-e9812163d25b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:11.033590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5787f1e-b3d3-4475-b59e-e9812163d25b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:11.033822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5787f1e-b3d3-4475-b59e-e9812163d25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:11.038742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f3204-23e9-415e-bdc2-8c6f4e1799a3 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:11.070063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f3204-23e9-415e-bdc2-8c6f4e1799a3 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:11.084404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b5787f1e-b3d3-4475-b59e-e9812163d25b None None] MFA Rules not processed for user `27cc091f3b094cb1a1e2650a3fbd7671`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:11.086422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b5787f1e-b3d3-4475-b59e-e9812163d25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:11.101982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760d3fa-87ac-4876-8de7-7ba5b991490c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:11.102609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760d3fa-87ac-4876-8de7-7ba5b991490c service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:11.102900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760d3fa-87ac-4876-8de7-7ba5b991490c service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:11.103172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760d3fa-87ac-4876-8de7-7ba5b991490c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:11.106304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8760d3fa-87ac-4876-8de7-7ba5b991490c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:11.109351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 837/1692] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:03:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:11.121317 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5787f1e-b3d3-4475-b59e-e9812163d25b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:11.129055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b5787f1e-b3d3-4475-b59e-e9812163d25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:11.143159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760d3fa-87ac-4876-8de7-7ba5b991490c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:11.145189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5787f1e-b3d3-4475-b59e-e9812163d25b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:11.148678 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760d3fa-87ac-4876-8de7-7ba5b991490c service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:11.153010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73d15d-52e7-4163-894e-e0ebb726b9b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:11.153931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73d15d-52e7-4163-894e-e0ebb726b9b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:11.154647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73d15d-52e7-4163-894e-e0ebb726b9b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:11.155937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73d15d-52e7-4163-894e-e0ebb726b9b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:11.161724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 856/1693] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:11.181297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73d15d-52e7-4163-894e-e0ebb726b9b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:11.182882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73d15d-52e7-4163-894e-e0ebb726b9b4 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:11.198722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 838/1694] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:12.416025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c88eb-c04c-4d0e-a77a-6f02dc1ff9b9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:12.416705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c88eb-c04c-4d0e-a77a-6f02dc1ff9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:12.416891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c88eb-c04c-4d0e-a77a-6f02dc1ff9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:12.467758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d81c88eb-c04c-4d0e-a77a-6f02dc1ff9b9 None None] MFA Rules not processed for user `46c01f8b99e24be5b6c2d73b9b1bd9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:12.476620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d81c88eb-c04c-4d0e-a77a-6f02dc1ff9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:12.523542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 857/1695] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:14.562782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73d15d-52e7-4163-894e-e0ebb726b9b4 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:14.566328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bd73d15d-52e7-4163-894e-e0ebb726b9b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:14.675931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:14.729572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73d15d-52e7-4163-894e-e0ebb726b9b4 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:14.749353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:14.753600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:14.755050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:14.755536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] PATH_INFO: `/v3/users/1b1e2cf23d2f4167aec1221848a5f832` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:14.758691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b1e2cf23d2f4167aec1221848a5f832)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:14.769314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:14.864350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b1e2cf23d2f4167aec1221848a5f832'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.865942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b1e2cf23d2f4167aec1221848a5f832'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.867179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b1e2cf23d2f4167aec1221848a5f832'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.884780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b1e2cf23d2f4167aec1221848a5f832'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.906113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b1e2cf23d2f4167aec1221848a5f832'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.950642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 839/1696] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:14 2022] DELETE /identity/v3/users/1b1e2cf23d2f4167aec1221848a5f832 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:14.954758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba2492-edf4-4df0-ae14-b15020c35d52 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:14.955267 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba2492-edf4-4df0-ae14-b15020c35d52 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:14.955498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba2492-edf4-4df0-ae14-b15020c35d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:14.994779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-efba2492-edf4-4df0-ae14-b15020c35d52 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:15.000710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-efba2492-edf4-4df0-ae14-b15020c35d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:15.184350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:15.184350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:15.184708 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 858/1697] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:15.400816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf74c2-271a-4e90-8e9f-dc88f2a9f088 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:15.420764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d83ad-f832-4f88-a26d-27685640b4ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:15.421552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d83ad-f832-4f88-a26d-27685640b4ab service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:15.424894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d83ad-f832-4f88-a26d-27685640b4ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:15.428811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d83ad-f832-4f88-a26d-27685640b4ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:15.470889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d83ad-f832-4f88-a26d-27685640b4ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:15.472391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d83ad-f832-4f88-a26d-27685640b4ab service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:15.507062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 840/1698] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:16.480836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4b550-68e3-4d8e-94d9-a5aaa0f2cf68 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:16.481337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4b550-68e3-4d8e-94d9-a5aaa0f2cf68 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:16.481337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4b550-68e3-4d8e-94d9-a5aaa0f2cf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:16.528978 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d83ad-f832-4f88-a26d-27685640b4ab service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:16.528978 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9d1d83ad-f832-4f88-a26d-27685640b4ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:16.555205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d83ad-f832-4f88-a26d-27685640b4ab service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:16.576308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:16.576962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:16.577246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:16.577527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] PATH_INFO: `/v3/users/2e5d00edb03c4b6c91290f1d4bf4c28c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:16.584630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5d00edb03c4b6c91290f1d4bf4c28c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:16.586717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:16.625571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2ce4b550-68e3-4d8e-94d9-a5aaa0f2cf68 None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:16.627011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2ce4b550-68e3-4d8e-94d9-a5aaa0f2cf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:16.751669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5d00edb03c4b6c91290f1d4bf4c28c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:16.755405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5d00edb03c4b6c91290f1d4bf4c28c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:16.756562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5d00edb03c4b6c91290f1d4bf4c28c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:16.760782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 859/1699] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:03:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:16.793370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5d00edb03c4b6c91290f1d4bf4c28c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:16.811102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5d00edb03c4b6c91290f1d4bf4c28c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:16.812669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:16.886422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 841/1700] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:16 2022] DELETE /identity/v3/users/2e5d00edb03c4b6c91290f1d4bf4c28c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:16.891896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e5093-0f58-4971-ada8-32f1a776f57d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:16.892221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e5093-0f58-4971-ada8-32f1a776f57d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:16.893132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e5093-0f58-4971-ada8-32f1a776f57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:16.936559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-de4e5093-0f58-4971-ada8-32f1a776f57d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:16.937980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-de4e5093-0f58-4971-ada8-32f1a776f57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:17.009508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:17.011393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:17.091027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:17.178052 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 860/1701] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:17.179099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e5093-0f58-4971-ada8-32f1a776f57d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:17.182352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-de4e5093-0f58-4971-ada8-32f1a776f57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:17.207545 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e5093-0f58-4971-ada8-32f1a776f57d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:17.220219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:17.220880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:17.221161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:17.221449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] PATH_INFO: `/v3/users/ca01a580c6724762b61440711b94fbf7` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:17.224793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca01a580c6724762b61440711b94fbf7)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:17.225845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: WARNING py.warnings [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:03:17.225845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: warnings.warn(msg) Aug 24 10:03:17.225845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: Aug 24 10:03:17.227077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:17.233462 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a047d-e1e9-4d35-b5da-f2aee2e91b3f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:17.240984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8742-fe05-4064-862a-6fc5e35faadb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:17.243123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8742-fe05-4064-862a-6fc5e35faadb service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:17.244266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8742-fe05-4064-862a-6fc5e35faadb service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:17.245174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8742-fe05-4064-862a-6fc5e35faadb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:17.251512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6e2d8742-fe05-4064-862a-6fc5e35faadb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:17.388444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8742-fe05-4064-862a-6fc5e35faadb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:17.390135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8742-fe05-4064-862a-6fc5e35faadb service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:17.408760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca01a580c6724762b61440711b94fbf7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:17.410912 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca01a580c6724762b61440711b94fbf7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:17.413112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca01a580c6724762b61440711b94fbf7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:17.415428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 842/1702] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:03:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:03:17.416629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8742-fe05-4064-862a-6fc5e35faadb service swift] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:17.418526 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6e2d8742-fe05-4064-862a-6fc5e35faadb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:17.437453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca01a580c6724762b61440711b94fbf7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:17.474305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b7f6b725-5121-40c2-8d66-5200b5fc4418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca01a580c6724762b61440711b94fbf7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:17.535650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 861/1703] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:17 2022] DELETE /identity/v3/users/ca01a580c6724762b61440711b94fbf7 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:17.543549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7334c0e-0aed-4ff1-8cac-cf8fdadf1a12 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:17.543842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7334c0e-0aed-4ff1-8cac-cf8fdadf1a12 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:17.544143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7334c0e-0aed-4ff1-8cac-cf8fdadf1a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:17.585895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b7334c0e-0aed-4ff1-8cac-cf8fdadf1a12 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:17.587386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b7334c0e-0aed-4ff1-8cac-cf8fdadf1a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:17.636840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8742-fe05-4064-862a-6fc5e35faadb service swift] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:17.664601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:17.726507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 862/1704] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:17.760483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7334c0e-0aed-4ff1-8cac-cf8fdadf1a12 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:17.762423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b7334c0e-0aed-4ff1-8cac-cf8fdadf1a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:17.829072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad68e52-199d-4ed3-919c-0cd4e05e56a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:17.829698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad68e52-199d-4ed3-919c-0cd4e05e56a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:17.829977 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad68e52-199d-4ed3-919c-0cd4e05e56a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:17.830496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad68e52-199d-4ed3-919c-0cd4e05e56a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:17.840395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1ad68e52-199d-4ed3-919c-0cd4e05e56a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:17.877444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad68e52-199d-4ed3-919c-0cd4e05e56a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:17.879236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad68e52-199d-4ed3-919c-0cd4e05e56a5 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:17.899027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 843/1705] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:17.905922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:17.920817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7334c0e-0aed-4ff1-8cac-cf8fdadf1a12 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:17.940175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538204c-008f-46f8-ade8-1bc09d1cd461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:17.942239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538204c-008f-46f8-ade8-1bc09d1cd461 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:17.944138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538204c-008f-46f8-ade8-1bc09d1cd461 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:17.944510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538204c-008f-46f8-ade8-1bc09d1cd461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:17.975987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538204c-008f-46f8-ade8-1bc09d1cd461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:17.978498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538204c-008f-46f8-ade8-1bc09d1cd461 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:18.010360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 863/1706] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:18.014128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad68e52-199d-4ed3-919c-0cd4e05e56a5 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:18.015901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1ad68e52-199d-4ed3-919c-0cd4e05e56a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:18.039569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad68e52-199d-4ed3-919c-0cd4e05e56a5 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:18.052990 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:18.054248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.054864 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.055727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] PATH_INFO: `/v3/projects/80f07909b541427f857887bfe0426de5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:18.057440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80f07909b541427f857887bfe0426de5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:18.077174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:18.158370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80f07909b541427f857887bfe0426de5', 'request_id': 'req-256645da-46f4-4d93-b075-d5d636f70292'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:18.166511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80f07909b541427f857887bfe0426de5', 'request_id': 'req-256645da-46f4-4d93-b075-d5d636f70292'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:18.176257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80f07909b541427f857887bfe0426de5', 'request_id': 'req-256645da-46f4-4d93-b075-d5d636f70292'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:18.202261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] The token cache is being invalidate because project 80f07909b541427f857887bfe0426de5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:18.202729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:18.206249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-256645da-46f4-4d93-b075-d5d636f70292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:18.211905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 844/1707] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:18 2022] DELETE /identity/v3/projects/80f07909b541427f857887bfe0426de5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:18.215859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb207-d62e-4e4b-87e0-a805db2384c8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.217351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb207-d62e-4e4b-87e0-a805db2384c8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.217717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb207-d62e-4e4b-87e0-a805db2384c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:18.246573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-29cbb207-d62e-4e4b-87e0-a805db2384c8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:18.249016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-29cbb207-d62e-4e4b-87e0-a805db2384c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:18.375781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 864/1708] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:18.383413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713882-b152-40b5-8196-1d4c9c0dc138 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.383704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713882-b152-40b5-8196-1d4c9c0dc138 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.384228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713882-b152-40b5-8196-1d4c9c0dc138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:18.426513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d2713882-b152-40b5-8196-1d4c9c0dc138 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:18.427899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d2713882-b152-40b5-8196-1d4c9c0dc138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:18.449940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 845/1709] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:18.454723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbb207-d62e-4e4b-87e0-a805db2384c8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:18.465901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbb207-d62e-4e4b-87e0-a805db2384c8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:18.472491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ac81d-7600-498c-aac8-9baa84b814d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:18.473180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ac81d-7600-498c-aac8-9baa84b814d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.473528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ac81d-7600-498c-aac8-9baa84b814d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.473878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ac81d-7600-498c-aac8-9baa84b814d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:18.474610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ac81d-7600-498c-aac8-9baa84b814d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:18.474990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ac81d-7600-498c-aac8-9baa84b814d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:18.476762 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ac81d-7600-498c-aac8-9baa84b814d7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:18.506968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2713882-b152-40b5-8196-1d4c9c0dc138 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:18.508493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 865/1710] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:18.519760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2713882-b152-40b5-8196-1d4c9c0dc138 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:18.526346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2cae6-a4a2-40a3-9d78-2514d02db6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:18.527053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2cae6-a4a2-40a3-9d78-2514d02db6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.527785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2cae6-a4a2-40a3-9d78-2514d02db6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.527785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2cae6-a4a2-40a3-9d78-2514d02db6b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:18.528457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2cae6-a4a2-40a3-9d78-2514d02db6b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:18.533758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2cae6-a4a2-40a3-9d78-2514d02db6b2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:18.594088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ac81d-7600-498c-aac8-9baa84b814d7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:18.595312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 846/1711] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:18 2022] POST /identity/v3/projects => generated 375 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:18.610405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ac81d-7600-498c-aac8-9baa84b814d7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:18.730569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392da9b-632c-4588-bd21-fa6dd5b6d589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:18.731667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392da9b-632c-4588-bd21-fa6dd5b6d589 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.732180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392da9b-632c-4588-bd21-fa6dd5b6d589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.732874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392da9b-632c-4588-bd21-fa6dd5b6d589 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:18.733711 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392da9b-632c-4588-bd21-fa6dd5b6d589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:18.737950 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392da9b-632c-4588-bd21-fa6dd5b6d589 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:18.775922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 866/1712] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:18 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:18.780760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2cae6-a4a2-40a3-9d78-2514d02db6b2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:18.795928 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2cae6-a4a2-40a3-9d78-2514d02db6b2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:18.802261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb911660-322f-43da-a101-7b327741447c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:18.803444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911660-322f-43da-a101-7b327741447c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.804151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911660-322f-43da-a101-7b327741447c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.804651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911660-322f-43da-a101-7b327741447c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:18.805765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb911660-322f-43da-a101-7b327741447c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:18.811369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb911660-322f-43da-a101-7b327741447c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:18.828159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 847/1713] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:18.832382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392da9b-632c-4588-bd21-fa6dd5b6d589 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:18.852359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392da9b-632c-4588-bd21-fa6dd5b6d589 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:18.859823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bca63a-df7a-4858-9b1a-6473515ccc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:18.861383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bca63a-df7a-4858-9b1a-6473515ccc4d admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.861669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bca63a-df7a-4858-9b1a-6473515ccc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.861942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bca63a-df7a-4858-9b1a-6473515ccc4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:18.862613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bca63a-df7a-4858-9b1a-6473515ccc4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:18.864346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bca63a-df7a-4858-9b1a-6473515ccc4d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:18.888561 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 867/1714] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:18.893802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb911660-322f-43da-a101-7b327741447c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:18.925402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb911660-322f-43da-a101-7b327741447c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:18.936708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f8eb4-cfee-4401-b804-ecd209f9c760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:18.937538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f8eb4-cfee-4401-b804-ecd209f9c760 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.937921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f8eb4-cfee-4401-b804-ecd209f9c760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.939810 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f8eb4-cfee-4401-b804-ecd209f9c760 admin admin] PATH_INFO: `/v3/projects/cb3ad0552e544f35b45820c96d033974/users/a5ce68d8ee3347cd93b8c240e8a3245c/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:18.984322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f8eb4-cfee-4401-b804-ecd209f9c760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3ad0552e544f35b45820c96d033974, user_id=a5ce68d8ee3347cd93b8c240e8a3245c, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:18.995361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f8eb4-cfee-4401-b804-ecd209f9c760 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:19.027343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 848/1715] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:18 2022] PUT /identity/v3/projects/cb3ad0552e544f35b45820c96d033974/users/a5ce68d8ee3347cd93b8c240e8a3245c/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:19.039089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bca63a-df7a-4858-9b1a-6473515ccc4d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:19.040147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae167f-d0d4-4c90-a398-5995ea4c3481 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:19.040507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae167f-d0d4-4c90-a398-5995ea4c3481 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:19.041242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae167f-d0d4-4c90-a398-5995ea4c3481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:19.041358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-08bca63a-df7a-4858-9b1a-6473515ccc4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:19.065208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bca63a-df7a-4858-9b1a-6473515ccc4d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:19.083439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d0ae167f-d0d4-4c90-a398-5995ea4c3481 None None] MFA Rules not processed for user `a5ce68d8ee3347cd93b8c240e8a3245c`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:19.085326 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d0ae167f-d0d4-4c90-a398-5995ea4c3481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:19.138756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:19.207010 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:19.227340 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:19.228002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:19.228410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:19.228696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] PATH_INFO: `/v3/projects/9e7aa26bd9e64b30a6c0c3d3824a1dda` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:19.230335 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7aa26bd9e64b30a6c0c3d3824a1dda)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:19.236278 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:19.243576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 849/1716] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:19.258441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae167f-d0d4-4c90-a398-5995ea4c3481 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:19.261320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d0ae167f-d0d4-4c90-a398-5995ea4c3481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:19.447706 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7aa26bd9e64b30a6c0c3d3824a1dda', 'request_id': 'req-42824cfb-16a3-472e-9863-7ea15a15aacf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:19.475455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7aa26bd9e64b30a6c0c3d3824a1dda', 'request_id': 'req-42824cfb-16a3-472e-9863-7ea15a15aacf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:19.487717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae167f-d0d4-4c90-a398-5995ea4c3481 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:19.493455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7aa26bd9e64b30a6c0c3d3824a1dda', 'request_id': 'req-42824cfb-16a3-472e-9863-7ea15a15aacf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:19.531623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] The token cache is being invalidate because project 9e7aa26bd9e64b30a6c0c3d3824a1dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:19.537296 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:19.539798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:19.546759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 868/1717] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:19 2022] DELETE /identity/v3/projects/9e7aa26bd9e64b30a6c0c3d3824a1dda => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:19.551093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:19.556096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:19.674901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2464391-1afd-4f91-92db-ab0ebd2e4e06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:19.675898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2464391-1afd-4f91-92db-ab0ebd2e4e06 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:19.676397 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2464391-1afd-4f91-92db-ab0ebd2e4e06 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:19.676863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2464391-1afd-4f91-92db-ab0ebd2e4e06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:19.691564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a2464391-1afd-4f91-92db-ab0ebd2e4e06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:19.750385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42824cfb-16a3-472e-9863-7ea15a15aacf admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:19.792141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:19.793175 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:19.793359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2464391-1afd-4f91-92db-ab0ebd2e4e06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:19.794551 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:19.795030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] PATH_INFO: `/v3/projects/abcce51c445b46fdab781056f45743cd` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:19.796148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2464391-1afd-4f91-92db-ab0ebd2e4e06 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:19.798820 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abcce51c445b46fdab781056f45743cd)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:19.804092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:19.810766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 850/1718] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:19.812848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13aca83-a62b-4389-97f5-496d882fc16e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:19.813165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13aca83-a62b-4389-97f5-496d882fc16e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:19.813690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13aca83-a62b-4389-97f5-496d882fc16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:19.857717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:19.907035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d13aca83-a62b-4389-97f5-496d882fc16e None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:19.908310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d13aca83-a62b-4389-97f5-496d882fc16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:19.959566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 851/1719] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:19.959774 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed118d8-046a-47a2-98b3-55d05f619bb9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:19.960596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed118d8-046a-47a2-98b3-55d05f619bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:19.963143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed118d8-046a-47a2-98b3-55d05f619bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:19.993151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abcce51c445b46fdab781056f45743cd', 'request_id': 'req-e6767f73-9163-47af-9d66-7b9e7128a9c6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:20.014603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abcce51c445b46fdab781056f45743cd', 'request_id': 'req-e6767f73-9163-47af-9d66-7b9e7128a9c6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:20.041534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abcce51c445b46fdab781056f45743cd', 'request_id': 'req-e6767f73-9163-47af-9d66-7b9e7128a9c6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:20.054869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0ed118d8-046a-47a2-98b3-55d05f619bb9 None None] MFA Rules not processed for user `27cc091f3b094cb1a1e2650a3fbd7671`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:20.056485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0ed118d8-046a-47a2-98b3-55d05f619bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:20.086116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] The token cache is being invalidate because project abcce51c445b46fdab781056f45743cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:20.086530 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:20.088780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e6767f73-9163-47af-9d66-7b9e7128a9c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:20.090379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 869/1720] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:19 2022] DELETE /identity/v3/projects/abcce51c445b46fdab781056f45743cd => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:20.091600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe40cb0-d9cf-44d1-a5cf-d6957ed53d49 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:20.091923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe40cb0-d9cf-44d1-a5cf-d6957ed53d49 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:20.092259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe40cb0-d9cf-44d1-a5cf-d6957ed53d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:20.148245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cbe40cb0-d9cf-44d1-a5cf-d6957ed53d49 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:20.153517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cbe40cb0-d9cf-44d1-a5cf-d6957ed53d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:20.202460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:20.244328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 852/1721] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 24 10:03:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:20.245373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed118d8-046a-47a2-98b3-55d05f619bb9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:20.246853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0ed118d8-046a-47a2-98b3-55d05f619bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:20.435007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed118d8-046a-47a2-98b3-55d05f619bb9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:20.435605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 870/1722] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:20.437294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe40cb0-d9cf-44d1-a5cf-d6957ed53d49 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:20.447764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:20.448330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:20.448642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:20.448886 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] PATH_INFO: `/v3/users/da187b84e25a4a369aacfba4c5a9aa60` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:20.450339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] RBAC: Authorizing `identity:delete_user(user_id=da187b84e25a4a369aacfba4c5a9aa60)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:20.452373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:20.458581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe40cb0-d9cf-44d1-a5cf-d6957ed53d49 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:20.468931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f6266-a7cc-43f6-84cb-cb0641683786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:20.472147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f6266-a7cc-43f6-84cb-cb0641683786 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:20.472811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f6266-a7cc-43f6-84cb-cb0641683786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:20.473253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f6266-a7cc-43f6-84cb-cb0641683786 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:20.474254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f6266-a7cc-43f6-84cb-cb0641683786 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:20.475104 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f6266-a7cc-43f6-84cb-cb0641683786 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:20.487114 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f6266-a7cc-43f6-84cb-cb0641683786 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:20.527960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 871/1723] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:20.544616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f6266-a7cc-43f6-84cb-cb0641683786 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:20.555348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cf3f6266-a7cc-43f6-84cb-cb0641683786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:20.570420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da187b84e25a4a369aacfba4c5a9aa60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:20.571109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da187b84e25a4a369aacfba4c5a9aa60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:20.571927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da187b84e25a4a369aacfba4c5a9aa60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:20.591573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f6266-a7cc-43f6-84cb-cb0641683786 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:20.597734 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b31661-3990-4772-a76b-64cf398b3ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:20.598155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da187b84e25a4a369aacfba4c5a9aa60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:20.598478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b31661-3990-4772-a76b-64cf398b3ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:20.598762 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b31661-3990-4772-a76b-64cf398b3ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:20.604143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b31661-3990-4772-a76b-64cf398b3ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:20.607574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-86b31661-3990-4772-a76b-64cf398b3ad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:20.618383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da187b84e25a4a369aacfba4c5a9aa60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:20.652517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b31661-3990-4772-a76b-64cf398b3ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:20.658170 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 853/1724] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:20 2022] DELETE /identity/v3/users/da187b84e25a4a369aacfba4c5a9aa60 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:20.659451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b31661-3990-4772-a76b-64cf398b3ad6 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:20.659982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:20.662837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:20.674641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 872/1725] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:20.677330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b31661-3990-4772-a76b-64cf398b3ad6 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:20.679078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-86b31661-3990-4772-a76b-64cf398b3ad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:20.696137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b31661-3990-4772-a76b-64cf398b3ad6 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:20.742504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:20.807326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c8fcc-53cc-4638-9547-06651eb1dad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:20.809030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f109f15-b704-49c6-adac-de36868f1ace admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:20.809490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c8fcc-53cc-4638-9547-06651eb1dad1 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:20.809645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c8fcc-53cc-4638-9547-06651eb1dad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:20.809933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c8fcc-53cc-4638-9547-06651eb1dad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:20.814970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d836-7bac-4c01-97d1-9cc063262f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:20.815785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d836-7bac-4c01-97d1-9cc063262f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:20.816267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d836-7bac-4c01-97d1-9cc063262f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:20.816692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d836-7bac-4c01-97d1-9cc063262f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:20.817612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8d836-7bac-4c01-97d1-9cc063262f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:20.819547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8d836-7bac-4c01-97d1-9cc063262f87 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:20.846780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c8fcc-53cc-4638-9547-06651eb1dad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:20.849472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c8fcc-53cc-4638-9547-06651eb1dad1 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:20.865761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 873/1726] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:20.867043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c8fcc-53cc-4638-9547-06651eb1dad1 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:20.870630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 854/1727] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:20 2022] POST /identity/v3/projects => generated 403 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:20.870630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d836-7bac-4c01-97d1-9cc063262f87 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:20.872450 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-936c8fcc-53cc-4638-9547-06651eb1dad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:20.872590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b5b8d836-7bac-4c01-97d1-9cc063262f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:20.891593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c8fcc-53cc-4638-9547-06651eb1dad1 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:21.044332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae2f8b-cbf9-4e4f-b3ae-1aeb2f2f9f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:21.046616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae2f8b-cbf9-4e4f-b3ae-1aeb2f2f9f73 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:21.048715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d836-7bac-4c01-97d1-9cc063262f87 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:21.048833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae2f8b-cbf9-4e4f-b3ae-1aeb2f2f9f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:21.049199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae2f8b-cbf9-4e4f-b3ae-1aeb2f2f9f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:21.063018 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd7377-aafd-495b-9109-e4081696ce41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:21.063831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7377-aafd-495b-9109-e4081696ce41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:21.064143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7377-aafd-495b-9109-e4081696ce41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:21.064425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7377-aafd-495b-9109-e4081696ce41 admin admin] PATH_INFO: `/v3/users/a5ce68d8ee3347cd93b8c240e8a3245c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:21.069101 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd7377-aafd-495b-9109-e4081696ce41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ce68d8ee3347cd93b8c240e8a3245c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:21.073324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd7377-aafd-495b-9109-e4081696ce41 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:21.079483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae2f8b-cbf9-4e4f-b3ae-1aeb2f2f9f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:21.081993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae2f8b-cbf9-4e4f-b3ae-1aeb2f2f9f73 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:21.102231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 874/1728] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:21.103270 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae2f8b-cbf9-4e4f-b3ae-1aeb2f2f9f73 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:21.174405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae2f8b-cbf9-4e4f-b3ae-1aeb2f2f9f73 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:21.190315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6cf6-479c-4b8e-85ab-6a41a70e167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:21.191688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6cf6-479c-4b8e-85ab-6a41a70e167e admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:21.192482 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6cf6-479c-4b8e-85ab-6a41a70e167e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:21.193973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6cf6-479c-4b8e-85ab-6a41a70e167e admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:21.194742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef6cf6-479c-4b8e-85ab-6a41a70e167e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:21.197266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef6cf6-479c-4b8e-85ab-6a41a70e167e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:21.248916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 875/1729] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:21 2022] POST /identity/v3/users => generated 302 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:21.252127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6cf6-479c-4b8e-85ab-6a41a70e167e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:21.268287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6cf6-479c-4b8e-85ab-6a41a70e167e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:21.274251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521a459-4f7b-4935-a789-1398a1efb9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:21.275316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521a459-4f7b-4935-a789-1398a1efb9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:21.276128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521a459-4f7b-4935-a789-1398a1efb9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:21.276612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521a459-4f7b-4935-a789-1398a1efb9c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:21.277683 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521a459-4f7b-4935-a789-1398a1efb9c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:21.279929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521a459-4f7b-4935-a789-1398a1efb9c2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:21.281135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eafd7377-aafd-495b-9109-e4081696ce41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ce68d8ee3347cd93b8c240e8a3245c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:21.282386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eafd7377-aafd-495b-9109-e4081696ce41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ce68d8ee3347cd93b8c240e8a3245c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:21.283277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eafd7377-aafd-495b-9109-e4081696ce41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ce68d8ee3347cd93b8c240e8a3245c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:21.307558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 876/1730] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:21.308119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eafd7377-aafd-495b-9109-e4081696ce41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ce68d8ee3347cd93b8c240e8a3245c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:21.321305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521a459-4f7b-4935-a789-1398a1efb9c2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:21.326670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4521a459-4f7b-4935-a789-1398a1efb9c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:21.336182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eafd7377-aafd-495b-9109-e4081696ce41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ce68d8ee3347cd93b8c240e8a3245c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:21.364447 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521a459-4f7b-4935-a789-1398a1efb9c2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:21.378774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945ce31-a474-413b-b30e-0bb77b042a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:21.379453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945ce31-a474-413b-b30e-0bb77b042a6d admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:21.379732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945ce31-a474-413b-b30e-0bb77b042a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:21.380009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945ce31-a474-413b-b30e-0bb77b042a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:21.380686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945ce31-a474-413b-b30e-0bb77b042a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:21.386341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945ce31-a474-413b-b30e-0bb77b042a6d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:21.393574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:21.412147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 855/1731] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:20 2022] DELETE /identity/v3/users/a5ce68d8ee3347cd93b8c240e8a3245c => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:21.415862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e801930-c358-487e-b0ab-b25cd3ec139b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:21.416235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e801930-c358-487e-b0ab-b25cd3ec139b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:21.416496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e801930-c358-487e-b0ab-b25cd3ec139b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:21.427196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 877/1732] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:21.429281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945ce31-a474-413b-b30e-0bb77b042a6d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:21.454528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945ce31-a474-413b-b30e-0bb77b042a6d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:21.463212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3e801930-c358-487e-b0ab-b25cd3ec139b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:21.467448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3e801930-c358-487e-b0ab-b25cd3ec139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:21.532459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:21.653108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca521b-8b97-47cb-9d90-3e886f85d08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:21.653739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 856/1733] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:21.654461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca521b-8b97-47cb-9d90-3e886f85d08e admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:21.655066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca521b-8b97-47cb-9d90-3e886f85d08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:21.656476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca521b-8b97-47cb-9d90-3e886f85d08e admin admin] PATH_INFO: `/v3/projects/5218b6e9c7664dc5b0716d6f551f9fab/users/7dffe1624c964a4db591bb702e1a23de/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:21.693454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca521b-8b97-47cb-9d90-3e886f85d08e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5218b6e9c7664dc5b0716d6f551f9fab, user_id=7dffe1624c964a4db591bb702e1a23de, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:21.696673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca521b-8b97-47cb-9d90-3e886f85d08e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:21.711160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e801930-c358-487e-b0ab-b25cd3ec139b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:21.713025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3e801930-c358-487e-b0ab-b25cd3ec139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:21.739931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 878/1734] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:21 2022] PUT /identity/v3/projects/5218b6e9c7664dc5b0716d6f551f9fab/users/7dffe1624c964a4db591bb702e1a23de/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:21.752277 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028454b-adbd-4cbc-a004-ec23f5b73f4c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:21.759825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028454b-adbd-4cbc-a004-ec23f5b73f4c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:21.760356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028454b-adbd-4cbc-a004-ec23f5b73f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:21.886335 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4028454b-adbd-4cbc-a004-ec23f5b73f4c None None] MFA Rules not processed for user `7dffe1624c964a4db591bb702e1a23de`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:21.889469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4028454b-adbd-4cbc-a004-ec23f5b73f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:21.970673 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e801930-c358-487e-b0ab-b25cd3ec139b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:22.097781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e052c-9e48-41a6-aa10-a49fd9d298ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:22.098657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e052c-9e48-41a6-aa10-a49fd9d298ea service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:22.099082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e052c-9e48-41a6-aa10-a49fd9d298ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:22.099500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e052c-9e48-41a6-aa10-a49fd9d298ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:22.205510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 879/1735] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:21 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:22.210180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028454b-adbd-4cbc-a004-ec23f5b73f4c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:22.280691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e052c-9e48-41a6-aa10-a49fd9d298ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:22.282224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e052c-9e48-41a6-aa10-a49fd9d298ea service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:22.321658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 857/1736] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:22.407859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028454b-adbd-4cbc-a004-ec23f5b73f4c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:22.422961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19145d59-c93f-48d4-baca-16f5ad21009c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:22.423638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19145d59-c93f-48d4-baca-16f5ad21009c admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:22.424220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19145d59-c93f-48d4-baca-16f5ad21009c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:22.424519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19145d59-c93f-48d4-baca-16f5ad21009c admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:22.425163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19145d59-c93f-48d4-baca-16f5ad21009c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:22.427741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19145d59-c93f-48d4-baca-16f5ad21009c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:22.460744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:22.484621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 880/1737] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:22 2022] POST /identity/v3/projects => generated 405 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:22.485683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e052c-9e48-41a6-aa10-a49fd9d298ea service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:22.501254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e052c-9e48-41a6-aa10-a49fd9d298ea service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:22.600070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:22.657563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5801f4f-37fc-4747-93ec-4da9bdedb2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:22.658506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5801f4f-37fc-4747-93ec-4da9bdedb2db admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:22.659458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5801f4f-37fc-4747-93ec-4da9bdedb2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:22.660334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5801f4f-37fc-4747-93ec-4da9bdedb2db admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:22.661326 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5801f4f-37fc-4747-93ec-4da9bdedb2db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:22.663582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5801f4f-37fc-4747-93ec-4da9bdedb2db admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:22.700020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 858/1738] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:22 2022] POST /identity/v3/users => generated 309 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:22.704968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19145d59-c93f-48d4-baca-16f5ad21009c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:22.732298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19145d59-c93f-48d4-baca-16f5ad21009c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:22.768305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fdb16-bd58-4959-9878-0d8d27656803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:22.769727 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fdb16-bd58-4959-9878-0d8d27656803 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:22.769851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fdb16-bd58-4959-9878-0d8d27656803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:22.772082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fdb16-bd58-4959-9878-0d8d27656803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:22.775435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fdb16-bd58-4959-9878-0d8d27656803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:22.778988 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fdb16-bd58-4959-9878-0d8d27656803 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:22.810753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 881/1739] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:22.815498 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5801f4f-37fc-4747-93ec-4da9bdedb2db admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:22.839868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5801f4f-37fc-4747-93ec-4da9bdedb2db admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:22.861185 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dd4ec-f4c9-4810-a0a0-dd05f40e71bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:22.861862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dd4ec-f4c9-4810-a0a0-dd05f40e71bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:22.863383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dd4ec-f4c9-4810-a0a0-dd05f40e71bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:22.871202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dd4ec-f4c9-4810-a0a0-dd05f40e71bc admin admin] PATH_INFO: `/v3/projects/314d41e4de2b48ffa65400e05a59854f/users/a5a482f5d0b3472faae96eb35bfc4db1/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:22.942783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573dd4ec-f4c9-4810-a0a0-dd05f40e71bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=314d41e4de2b48ffa65400e05a59854f, user_id=a5a482f5d0b3472faae96eb35bfc4db1, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:22.945813 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573dd4ec-f4c9-4810-a0a0-dd05f40e71bc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:22.973098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 859/1740] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:22 2022] PUT /identity/v3/projects/314d41e4de2b48ffa65400e05a59854f/users/a5a482f5d0b3472faae96eb35bfc4db1/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:22.976946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fdb16-bd58-4959-9878-0d8d27656803 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:22.987050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dd4ec-f4c9-4810-a0a0-dd05f40e71bc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:22.989336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-573dd4ec-f4c9-4810-a0a0-dd05f40e71bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:22.993493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fdb16-bd58-4959-9878-0d8d27656803 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:23.105448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dd4ec-f4c9-4810-a0a0-dd05f40e71bc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:23.126274 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:23.126932 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.127359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.127488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] PATH_INFO: `/v3/projects/bcf7710287bd41e99028493910906a24` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.129329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcf7710287bd41e99028493910906a24)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:23.131523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:23.170556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf95116-51aa-407b-b5a2-47f5bf57db5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:23.174157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf95116-51aa-407b-b5a2-47f5bf57db5b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.174989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf95116-51aa-407b-b5a2-47f5bf57db5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.175470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf95116-51aa-407b-b5a2-47f5bf57db5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.176780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf95116-51aa-407b-b5a2-47f5bf57db5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:23.178628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf95116-51aa-407b-b5a2-47f5bf57db5b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:23.198618 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 882/1741] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:23.202423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf95116-51aa-407b-b5a2-47f5bf57db5b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:23.209573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcf7710287bd41e99028493910906a24', 'request_id': 'req-969edb36-c0e5-4f8b-b30c-49c573e7f84d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:23.214634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf95116-51aa-407b-b5a2-47f5bf57db5b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:23.220442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcf7710287bd41e99028493910906a24', 'request_id': 'req-969edb36-c0e5-4f8b-b30c-49c573e7f84d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:23.237649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcf7710287bd41e99028493910906a24', 'request_id': 'req-969edb36-c0e5-4f8b-b30c-49c573e7f84d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:23.265806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] The token cache is being invalidate because project bcf7710287bd41e99028493910906a24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:23.266146 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:23.276368 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-969edb36-c0e5-4f8b-b30c-49c573e7f84d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:23.279701 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 860/1742] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:22 2022] DELETE /identity/v3/projects/bcf7710287bd41e99028493910906a24 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:23.286049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13495249-1b92-46d3-bb8a-10f66b302187 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.286336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13495249-1b92-46d3-bb8a-10f66b302187 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.287145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13495249-1b92-46d3-bb8a-10f66b302187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.317772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25348b3e-d841-44ac-98cd-4845741891aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:23.317985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-13495249-1b92-46d3-bb8a-10f66b302187 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:23.320087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-13495249-1b92-46d3-bb8a-10f66b302187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:23.323676 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25348b3e-d841-44ac-98cd-4845741891aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.324822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25348b3e-d841-44ac-98cd-4845741891aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.325565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25348b3e-d841-44ac-98cd-4845741891aa admin admin] PATH_INFO: `/v3/projects/314d41e4de2b48ffa65400e05a59854f/users/a5a482f5d0b3472faae96eb35bfc4db1/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.328113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25348b3e-d841-44ac-98cd-4845741891aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=314d41e4de2b48ffa65400e05a59854f, user_id=a5a482f5d0b3472faae96eb35bfc4db1, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:23.329331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 861/1743] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:23.331494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25348b3e-d841-44ac-98cd-4845741891aa admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:23.338436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf71f-2d84-4885-9ead-b8d8af10ca5c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.338757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf71f-2d84-4885-9ead-b8d8af10ca5c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.339144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf71f-2d84-4885-9ead-b8d8af10ca5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.358062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 883/1744] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:23 2022] PUT /identity/v3/projects/314d41e4de2b48ffa65400e05a59854f/users/a5a482f5d0b3472faae96eb35bfc4db1/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:23.364088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbc260-d6ea-46ce-ab80-8a4076db116a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.364088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbc260-d6ea-46ce-ab80-8a4076db116a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.364359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbc260-d6ea-46ce-ab80-8a4076db116a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.386573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-164cf71f-2d84-4885-9ead-b8d8af10ca5c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:23.387824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-164cf71f-2d84-4885-9ead-b8d8af10ca5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:23.397025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-54cbc260-d6ea-46ce-ab80-8a4076db116a None None] MFA Rules not processed for user `a5a482f5d0b3472faae96eb35bfc4db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:23.398525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-54cbc260-d6ea-46ce-ab80-8a4076db116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:23.432424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:23.507887 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 862/1745] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:23.509144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf71f-2d84-4885-9ead-b8d8af10ca5c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:23.519893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf71f-2d84-4885-9ead-b8d8af10ca5c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:23.525513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514bc89-5145-4359-ae21-1242181afb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:23.528098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514bc89-5145-4359-ae21-1242181afb66 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.528528 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514bc89-5145-4359-ae21-1242181afb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.529520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514bc89-5145-4359-ae21-1242181afb66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.530335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514bc89-5145-4359-ae21-1242181afb66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:23.530916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514bc89-5145-4359-ae21-1242181afb66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:23.531462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:23.534930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514bc89-5145-4359-ae21-1242181afb66 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:23.536885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 884/1746] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:23 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:23.548672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbc260-d6ea-46ce-ab80-8a4076db116a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:23.550491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-54cbc260-d6ea-46ce-ab80-8a4076db116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:23.560210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 863/1747] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:23.561282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514bc89-5145-4359-ae21-1242181afb66 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:23.572337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514bc89-5145-4359-ae21-1242181afb66 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:23.579330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d4efb-a981-463c-8a90-e6c6e2b3f3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:23.580115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4efb-a981-463c-8a90-e6c6e2b3f3d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.580562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4efb-a981-463c-8a90-e6c6e2b3f3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.580946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4efb-a981-463c-8a90-e6c6e2b3f3d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.581737 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d4efb-a981-463c-8a90-e6c6e2b3f3d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:23.594654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d4efb-a981-463c-8a90-e6c6e2b3f3d9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:23.652649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbc260-d6ea-46ce-ab80-8a4076db116a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:23.658228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b806aaf-7504-4668-95c9-bc4581f375cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:23.659638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b806aaf-7504-4668-95c9-bc4581f375cd service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.660085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b806aaf-7504-4668-95c9-bc4581f375cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.660516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b806aaf-7504-4668-95c9-bc4581f375cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.666132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 864/1748] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:23 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:23.669299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82a3b9-7a6f-4e10-96a3-652ca4bf04d7 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.669586 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82a3b9-7a6f-4e10-96a3-652ca4bf04d7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.669864 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82a3b9-7a6f-4e10-96a3-652ca4bf04d7 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.671293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 865/1749] 10.222.0.86 () {54 vars in 921 bytes} [Wed Aug 24 10:03:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:03:23.672057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82a3b9-7a6f-4e10-96a3-652ca4bf04d7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:23.687089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b806aaf-7504-4668-95c9-bc4581f375cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:23.688646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b806aaf-7504-4668-95c9-bc4581f375cd service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:23.712082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 885/1750] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:23 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:23.714686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c90e1e-115e-4ff2-b0dc-26f06f4ed314 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.715031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c90e1e-115e-4ff2-b0dc-26f06f4ed314 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.715362 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c90e1e-115e-4ff2-b0dc-26f06f4ed314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.739767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c8c90e1e-115e-4ff2-b0dc-26f06f4ed314 None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:23.740949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c8c90e1e-115e-4ff2-b0dc-26f06f4ed314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:23.837600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82a3b9-7a6f-4e10-96a3-652ca4bf04d7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:23.850234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d48eb2-5a4c-4ef2-92ce-5fa0a59fc425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:23.852629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d48eb2-5a4c-4ef2-92ce-5fa0a59fc425 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.853030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d48eb2-5a4c-4ef2-92ce-5fa0a59fc425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.853422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d48eb2-5a4c-4ef2-92ce-5fa0a59fc425 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.855145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d48eb2-5a4c-4ef2-92ce-5fa0a59fc425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:23.858865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d48eb2-5a4c-4ef2-92ce-5fa0a59fc425 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:23.906490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 866/1751] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:23 2022] POST /identity/v3/users => generated 283 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:23.909845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d48eb2-5a4c-4ef2-92ce-5fa0a59fc425 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:23.914783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-70d48eb2-5a4c-4ef2-92ce-5fa0a59fc425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:23.915533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 886/1752] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Aug 24 10:03:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:23.917143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c90e1e-115e-4ff2-b0dc-26f06f4ed314 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:23.938862 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c90e1e-115e-4ff2-b0dc-26f06f4ed314 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:23.942862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d48eb2-5a4c-4ef2-92ce-5fa0a59fc425 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:23.951368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e00a8-349e-49b6-b92b-bebebef622b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:23.952646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e00a8-349e-49b6-b92b-bebebef622b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.953197 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e00a8-349e-49b6-b92b-bebebef622b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.954372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e00a8-349e-49b6-b92b-bebebef622b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.955315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e00a8-349e-49b6-b92b-bebebef622b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:23.957417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e00a8-349e-49b6-b92b-bebebef622b7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:23.961536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:23.963932 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:23.964465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:23.964896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] PATH_INFO: `/v3/projects/cb3ad0552e544f35b45820c96d033974` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:23.969799 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3ad0552e544f35b45820c96d033974)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:23.971983 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 887/1753] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:23.973095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e00a8-349e-49b6-b92b-bebebef622b7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:23.975651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f82e00a8-349e-49b6-b92b-bebebef622b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:23.978697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:23.995789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e00a8-349e-49b6-b92b-bebebef622b7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.003721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bdd71-b2e6-4ef8-aecd-ca2d1766a8cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.004495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bdd71-b2e6-4ef8-aecd-ca2d1766a8cf service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.004891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bdd71-b2e6-4ef8-aecd-ca2d1766a8cf service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.005277 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bdd71-b2e6-4ef8-aecd-ca2d1766a8cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.015486 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bdd71-b2e6-4ef8-aecd-ca2d1766a8cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.018078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bdd71-b2e6-4ef8-aecd-ca2d1766a8cf service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.032492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 888/1754] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:24.033788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bdd71-b2e6-4ef8-aecd-ca2d1766a8cf service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.046830 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bdd71-b2e6-4ef8-aecd-ca2d1766a8cf service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.053171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0450557-0700-4d48-aa84-dc68ec38a2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.055341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0450557-0700-4d48-aa84-dc68ec38a2ae admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.055982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0450557-0700-4d48-aa84-dc68ec38a2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.056426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0450557-0700-4d48-aa84-dc68ec38a2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.057298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0450557-0700-4d48-aa84-dc68ec38a2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.059502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0450557-0700-4d48-aa84-dc68ec38a2ae admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.085367 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3ad0552e544f35b45820c96d033974', 'request_id': 'req-9347dd91-61f2-426d-9509-93d85794591f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:24.087565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 889/1755] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:24.092895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3ad0552e544f35b45820c96d033974', 'request_id': 'req-9347dd91-61f2-426d-9509-93d85794591f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:24.095928 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df305d-1bbf-4e63-81d6-9b022eb4d493 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.096267 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df305d-1bbf-4e63-81d6-9b022eb4d493 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.098668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df305d-1bbf-4e63-81d6-9b022eb4d493 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.101947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 890/1756] 10.222.0.86 () {54 vars in 921 bytes} [Wed Aug 24 10:03:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:03:24.102742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df305d-1bbf-4e63-81d6-9b022eb4d493 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.110430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3ad0552e544f35b45820c96d033974', 'request_id': 'req-9347dd91-61f2-426d-9509-93d85794591f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:24.130943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] The token cache is being invalidate because project cb3ad0552e544f35b45820c96d033974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:24.131472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:24.133012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9347dd91-61f2-426d-9509-93d85794591f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:24.134868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 867/1757] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:23 2022] DELETE /identity/v3/projects/cb3ad0552e544f35b45820c96d033974 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:24.136951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27cdb0-bcdc-4637-b37e-f473f9111205 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.143376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27cdb0-bcdc-4637-b37e-f473f9111205 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.144098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27cdb0-bcdc-4637-b37e-f473f9111205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.162094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:24.189411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fb27cdb0-bcdc-4637-b37e-f473f9111205 None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:24.192024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fb27cdb0-bcdc-4637-b37e-f473f9111205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:24.244188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df305d-1bbf-4e63-81d6-9b022eb4d493 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.251354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c6d0-a758-4b18-a71d-3b25705690f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.252151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c6d0-a758-4b18-a71d-3b25705690f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.252437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c6d0-a758-4b18-a71d-3b25705690f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.252713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c6d0-a758-4b18-a71d-3b25705690f0 admin admin] PATH_INFO: `/v3/projects/e8d67c4463784059b12bdc3f332adca5/users/7d75e0f4dbc940a7aa394a3ab60f5b82/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.255749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:24.273249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351c6d0-a758-4b18-a71d-3b25705690f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d67c4463784059b12bdc3f332adca5, user_id=7d75e0f4dbc940a7aa394a3ab60f5b82, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.276330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351c6d0-a758-4b18-a71d-3b25705690f0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.283297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 868/1758] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Aug 24 10:03:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:24.285053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecf089-d6da-442a-ae6c-47b29cd43fab None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.285439 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecf089-d6da-442a-ae6c-47b29cd43fab None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.285826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecf089-d6da-442a-ae6c-47b29cd43fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.316555 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7aecf089-d6da-442a-ae6c-47b29cd43fab None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:24.318196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7aecf089-d6da-442a-ae6c-47b29cd43fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:24.323177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 891/1759] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:24 2022] PUT /identity/v3/projects/e8d67c4463784059b12bdc3f332adca5/users/7d75e0f4dbc940a7aa394a3ab60f5b82/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:24.327031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c6d0-a758-4b18-a71d-3b25705690f0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.330261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1351c6d0-a758-4b18-a71d-3b25705690f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:24.447230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 869/1760] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:24.448662 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a59a76-459b-4cdc-bae3-1bd0352ca85b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.448961 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a59a76-459b-4cdc-bae3-1bd0352ca85b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.449713 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a59a76-459b-4cdc-bae3-1bd0352ca85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.483545 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a8a59a76-459b-4cdc-bae3-1bd0352ca85b None None] MFA Rules not processed for user `7d75e0f4dbc940a7aa394a3ab60f5b82`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:24.486764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a8a59a76-459b-4cdc-bae3-1bd0352ca85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:24.492176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c6d0-a758-4b18-a71d-3b25705690f0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.498202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7055620-c2ac-4fe9-acb6-0ea1f90435ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.498803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7055620-c2ac-4fe9-acb6-0ea1f90435ae service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.499088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7055620-c2ac-4fe9-acb6-0ea1f90435ae service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.499368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7055620-c2ac-4fe9-acb6-0ea1f90435ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.510189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7055620-c2ac-4fe9-acb6-0ea1f90435ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.511791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7055620-c2ac-4fe9-acb6-0ea1f90435ae service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.522653 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 892/1761] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:24.523951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37724434-46c5-4081-9414-efe8f93a4dd9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.524222 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37724434-46c5-4081-9414-efe8f93a4dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.524499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37724434-46c5-4081-9414-efe8f93a4dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.545048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-37724434-46c5-4081-9414-efe8f93a4dd9 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:24.546245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-37724434-46c5-4081-9414-efe8f93a4dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:24.555797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 893/1762] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:24.560170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37724434-46c5-4081-9414-efe8f93a4dd9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.570881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37724434-46c5-4081-9414-efe8f93a4dd9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.574158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 870/1763] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:24.577550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e52f96-2efb-4894-8e6a-682596b4df3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.578236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e52f96-2efb-4894-8e6a-682596b4df3f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.578627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a59a76-459b-4cdc-bae3-1bd0352ca85b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.578743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e52f96-2efb-4894-8e6a-682596b4df3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.579045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e52f96-2efb-4894-8e6a-682596b4df3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.579694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e52f96-2efb-4894-8e6a-682596b4df3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:24.579995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e52f96-2efb-4894-8e6a-682596b4df3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.580121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a8a59a76-459b-4cdc-bae3-1bd0352ca85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:24.581775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e52f96-2efb-4894-8e6a-682596b4df3f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.594770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a59a76-459b-4cdc-bae3-1bd0352ca85b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.595606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 894/1764] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:24.599495 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1af50-f889-4aff-bd99-e42da59c17a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.599775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e52f96-2efb-4894-8e6a-682596b4df3f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.600150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1af50-f889-4aff-bd99-e42da59c17a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.600448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1af50-f889-4aff-bd99-e42da59c17a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.600759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1af50-f889-4aff-bd99-e42da59c17a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.601391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1af50-f889-4aff-bd99-e42da59c17a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.603350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1af50-f889-4aff-bd99-e42da59c17a7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.610043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e52f96-2efb-4894-8e6a-682596b4df3f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.615068 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88053d-6320-4dbf-923c-afcc02e40410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.615663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88053d-6320-4dbf-923c-afcc02e40410 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.615943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88053d-6320-4dbf-923c-afcc02e40410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.616256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88053d-6320-4dbf-923c-afcc02e40410 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.616881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88053d-6320-4dbf-923c-afcc02e40410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.618784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88053d-6320-4dbf-923c-afcc02e40410 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.632593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 871/1765] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:24 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:24.636497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1af50-f889-4aff-bd99-e42da59c17a7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.643036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:24.654271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 895/1766] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:24 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:24.658933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88053d-6320-4dbf-923c-afcc02e40410 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.716934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1af50-f889-4aff-bd99-e42da59c17a7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.752682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88053d-6320-4dbf-923c-afcc02e40410 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.760737 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405d328-ab03-4b8a-84fd-67f2847a87db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.761517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405d328-ab03-4b8a-84fd-67f2847a87db admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.761998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405d328-ab03-4b8a-84fd-67f2847a87db admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.762422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405d328-ab03-4b8a-84fd-67f2847a87db admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.763188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405d328-ab03-4b8a-84fd-67f2847a87db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.765720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405d328-ab03-4b8a-84fd-67f2847a87db admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.789755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269d68c-0801-4718-b631-b02110f45e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.791044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269d68c-0801-4718-b631-b02110f45e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.791044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269d68c-0801-4718-b631-b02110f45e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.791352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 896/1767] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:24 2022] POST /identity/v3/users => generated 285 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:24.791443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269d68c-0801-4718-b631-b02110f45e15 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.791875 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269d68c-0801-4718-b631-b02110f45e15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.793905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269d68c-0801-4718-b631-b02110f45e15 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.795010 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405d328-ab03-4b8a-84fd-67f2847a87db admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.808696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405d328-ab03-4b8a-84fd-67f2847a87db admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.817976 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511937e-e123-4bda-9337-58c4cc675dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.818803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 872/1768] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:24 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:24.820173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511937e-e123-4bda-9337-58c4cc675dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.820546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511937e-e123-4bda-9337-58c4cc675dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.820890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511937e-e123-4bda-9337-58c4cc675dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.821675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511937e-e123-4bda-9337-58c4cc675dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.823528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511937e-e123-4bda-9337-58c4cc675dbf admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.824513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269d68c-0801-4718-b631-b02110f45e15 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.841950 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 897/1769] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:24.843181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269d68c-0801-4718-b631-b02110f45e15 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.843319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511937e-e123-4bda-9337-58c4cc675dbf admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.848768 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb04183-c193-4995-aa11-e77b83de1c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.849387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb04183-c193-4995-aa11-e77b83de1c47 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.849655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb04183-c193-4995-aa11-e77b83de1c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.849939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb04183-c193-4995-aa11-e77b83de1c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.850576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb04183-c193-4995-aa11-e77b83de1c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.852470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb04183-c193-4995-aa11-e77b83de1c47 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.852869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:24.858481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511937e-e123-4bda-9337-58c4cc675dbf admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.866050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 873/1770] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:24.867972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada84b8-16ce-4254-9a31-aa77801e0107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.868614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada84b8-16ce-4254-9a31-aa77801e0107 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.868893 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada84b8-16ce-4254-9a31-aa77801e0107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.869189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada84b8-16ce-4254-9a31-aa77801e0107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.869810 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada84b8-16ce-4254-9a31-aa77801e0107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.870313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb04183-c193-4995-aa11-e77b83de1c47 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.871463 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada84b8-16ce-4254-9a31-aa77801e0107 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.881504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb04183-c193-4995-aa11-e77b83de1c47 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.886566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 898/1771] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:24.886894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cd3d2-b3a5-4cb5-9dd2-2fa242660d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.887454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada84b8-16ce-4254-9a31-aa77801e0107 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.887674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cd3d2-b3a5-4cb5-9dd2-2fa242660d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.888028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cd3d2-b3a5-4cb5-9dd2-2fa242660d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.888488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cd3d2-b3a5-4cb5-9dd2-2fa242660d9c admin admin] PATH_INFO: `/v3/projects/be338b2137bb448facb68975b9b3262a/users/8603306cc97b45d3aa2788cf0383bbf2/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.898356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada84b8-16ce-4254-9a31-aa77801e0107 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:24.904405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731102cd-473a-4fe2-bd7d-a488d09d9156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:24.905331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731102cd-473a-4fe2-bd7d-a488d09d9156 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.905616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731102cd-473a-4fe2-bd7d-a488d09d9156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.905893 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731102cd-473a-4fe2-bd7d-a488d09d9156 admin admin] PATH_INFO: `/v3/projects/13458a917c8044fbae06394f7d2b74e1/users/aea8f880ef934a96b51e2bdf1da92bc7/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.907362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670cd3d2-b3a5-4cb5-9dd2-2fa242660d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=be338b2137bb448facb68975b9b3262a, user_id=8603306cc97b45d3aa2788cf0383bbf2, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.910413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670cd3d2-b3a5-4cb5-9dd2-2fa242660d9c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.923024 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731102cd-473a-4fe2-bd7d-a488d09d9156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13458a917c8044fbae06394f7d2b74e1, user_id=aea8f880ef934a96b51e2bdf1da92bc7, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:24.924843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 874/1772] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:24 2022] PUT /identity/v3/projects/be338b2137bb448facb68975b9b3262a/users/8603306cc97b45d3aa2788cf0383bbf2/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:24.925991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731102cd-473a-4fe2-bd7d-a488d09d9156 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:24.928765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cd3d2-b3a5-4cb5-9dd2-2fa242660d9c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:24.939517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 899/1773] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:24 2022] PUT /identity/v3/projects/13458a917c8044fbae06394f7d2b74e1/users/aea8f880ef934a96b51e2bdf1da92bc7/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:24.949801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80051d-5cd9-4d90-bced-47b24cdd980c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:24.950111 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80051d-5cd9-4d90-bced-47b24cdd980c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:24.950390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80051d-5cd9-4d90-bced-47b24cdd980c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:24.984563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6c80051d-5cd9-4d90-bced-47b24cdd980c None None] MFA Rules not processed for user `aea8f880ef934a96b51e2bdf1da92bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:24.986457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6c80051d-5cd9-4d90-bced-47b24cdd980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:25.026260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cd3d2-b3a5-4cb5-9dd2-2fa242660d9c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:25.067058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:25.115997 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 900/1774] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:25.116984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80051d-5cd9-4d90-bced-47b24cdd980c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:25.118390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6c80051d-5cd9-4d90-bced-47b24cdd980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:25.190854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6adf-aa77-40f6-a889-791ad2007b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:25.191470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6adf-aa77-40f6-a889-791ad2007b62 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:25.191760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6adf-aa77-40f6-a889-791ad2007b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:25.192037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6adf-aa77-40f6-a889-791ad2007b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:25.192722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c6adf-aa77-40f6-a889-791ad2007b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:25.194569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c6adf-aa77-40f6-a889-791ad2007b62 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:25.203038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80051d-5cd9-4d90-bced-47b24cdd980c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:25.209170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:25.209908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:25.210186 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:25.210462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] PATH_INFO: `/v3/users/7dffe1624c964a4db591bb702e1a23de` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:25.210651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 875/1775] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:25.211497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6adf-aa77-40f6-a889-791ad2007b62 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:25.211751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dffe1624c964a4db591bb702e1a23de)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:25.214428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:25.223728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6adf-aa77-40f6-a889-791ad2007b62 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:25.229566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33a1e3-92ca-4124-80d7-f41218dfbe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:25.231131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33a1e3-92ca-4124-80d7-f41218dfbe77 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:25.231440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33a1e3-92ca-4124-80d7-f41218dfbe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:25.231703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33a1e3-92ca-4124-80d7-f41218dfbe77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:25.232388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33a1e3-92ca-4124-80d7-f41218dfbe77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:25.234320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33a1e3-92ca-4124-80d7-f41218dfbe77 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:25.272759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 876/1776] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:25 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:25.273614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33a1e3-92ca-4124-80d7-f41218dfbe77 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:25.331740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dffe1624c964a4db591bb702e1a23de'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:25.332721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dffe1624c964a4db591bb702e1a23de'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:25.333457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dffe1624c964a4db591bb702e1a23de'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:25.348213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dffe1624c964a4db591bb702e1a23de'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:25.356525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dffe1624c964a4db591bb702e1a23de'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:25.357940 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:25.409285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 901/1777] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:25 2022] DELETE /identity/v3/users/7dffe1624c964a4db591bb702e1a23de => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:25.410246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:25.411393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33a1e3-92ca-4124-80d7-f41218dfbe77 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:25.411982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:25.543427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e39a38-be31-4afa-af98-df0af81e36c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:25.544146 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e39a38-be31-4afa-af98-df0af81e36c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:25.544425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e39a38-be31-4afa-af98-df0af81e36c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:25.544790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e39a38-be31-4afa-af98-df0af81e36c1 admin admin] PATH_INFO: `/v3/projects/be338b2137bb448facb68975b9b3262a/users/8603306cc97b45d3aa2788cf0383bbf2/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:25.572312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e39a38-be31-4afa-af98-df0af81e36c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be338b2137bb448facb68975b9b3262a, user_id=8603306cc97b45d3aa2788cf0383bbf2, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:25.575717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e39a38-be31-4afa-af98-df0af81e36c1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:25.607619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 877/1778] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:25 2022] PUT /identity/v3/projects/be338b2137bb448facb68975b9b3262a/users/8603306cc97b45d3aa2788cf0383bbf2/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:25.609647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e1a68-8e55-4193-93f4-3dd18ad0586b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:25.610034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e1a68-8e55-4193-93f4-3dd18ad0586b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:25.610314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e1a68-8e55-4193-93f4-3dd18ad0586b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:25.643053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-288e1a68-8e55-4193-93f4-3dd18ad0586b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:25.644630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-288e1a68-8e55-4193-93f4-3dd18ad0586b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:25.668260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb17903-f7af-43c2-8fe1-51ace587f457 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:25.678380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:25.754933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 878/1779] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:25.756619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e79c-1895-4d8d-b4da-401cae3847b7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:25.757047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e79c-1895-4d8d-b4da-401cae3847b7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:25.757458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e79c-1895-4d8d-b4da-401cae3847b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:25.813721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e2360-151c-4887-b498-a9d9dfc0d959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:25.817707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e2360-151c-4887-b498-a9d9dfc0d959 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:25.818048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e2360-151c-4887-b498-a9d9dfc0d959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:25.818275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e2360-151c-4887-b498-a9d9dfc0d959 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:25.819557 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e2360-151c-4887-b498-a9d9dfc0d959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:25.822868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f6b6e79c-1895-4d8d-b4da-401cae3847b7 None None] MFA Rules not processed for user `8603306cc97b45d3aa2788cf0383bbf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:25.824791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e2360-151c-4887-b498-a9d9dfc0d959 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:25.826452 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f6b6e79c-1895-4d8d-b4da-401cae3847b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:25.886453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 902/1780] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:25 2022] POST /identity/v3/users => generated 290 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:25.889541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e2360-151c-4887-b498-a9d9dfc0d959 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:25.898970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-783e2360-151c-4887-b498-a9d9dfc0d959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:26.012494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 879/1781] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:26.015123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e79c-1895-4d8d-b4da-401cae3847b7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:26.048925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e79c-1895-4d8d-b4da-401cae3847b7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:26.060247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996e080-b70b-4724-9103-cc5a96ffa051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:26.061331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996e080-b70b-4724-9103-cc5a96ffa051 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:26.062067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996e080-b70b-4724-9103-cc5a96ffa051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:26.062480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996e080-b70b-4724-9103-cc5a96ffa051 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:26.063646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996e080-b70b-4724-9103-cc5a96ffa051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:26.065706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996e080-b70b-4724-9103-cc5a96ffa051 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:26.076233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e2360-151c-4887-b498-a9d9dfc0d959 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:26.080280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 880/1782] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:26.081432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996e080-b70b-4724-9103-cc5a96ffa051 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:26.084143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d996e080-b70b-4724-9103-cc5a96ffa051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:26.085887 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec2549-601f-4147-9230-72cbcc6a3f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:26.086607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec2549-601f-4147-9230-72cbcc6a3f7a service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:26.086938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec2549-601f-4147-9230-72cbcc6a3f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:26.087397 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec2549-601f-4147-9230-72cbcc6a3f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:26.109917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec2549-601f-4147-9230-72cbcc6a3f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:26.111879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec2549-601f-4147-9230-72cbcc6a3f7a service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:26.167394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 903/1783] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:26.170002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec2549-601f-4147-9230-72cbcc6a3f7a service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:26.207133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec2549-601f-4147-9230-72cbcc6a3f7a service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:26.226215 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996e080-b70b-4724-9103-cc5a96ffa051 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:26.227194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24976382-d1f4-47fa-8c06-478836b6f31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:26.228257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24976382-d1f4-47fa-8c06-478836b6f31e admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:26.229392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24976382-d1f4-47fa-8c06-478836b6f31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:26.237283 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24976382-d1f4-47fa-8c06-478836b6f31e admin admin] PATH_INFO: `/v3/projects/0c6dd87228164dcebe40a1a950f4f82a/users/227d1daba43c46da8bf2226d923cb87c/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:26.237469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fa9ab-45f5-4162-afdb-02205ec6555a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:26.238005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fa9ab-45f5-4162-afdb-02205ec6555a service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:26.238293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fa9ab-45f5-4162-afdb-02205ec6555a service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:26.238568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fa9ab-45f5-4162-afdb-02205ec6555a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:26.270119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fa9ab-45f5-4162-afdb-02205ec6555a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:26.273403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fa9ab-45f5-4162-afdb-02205ec6555a service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:26.294177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 881/1784] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:26.296811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24976382-d1f4-47fa-8c06-478836b6f31e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6dd87228164dcebe40a1a950f4f82a, user_id=227d1daba43c46da8bf2226d923cb87c, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:26.304572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24976382-d1f4-47fa-8c06-478836b6f31e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:26.311490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:26.342825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 904/1785] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:26 2022] PUT /identity/v3/projects/0c6dd87228164dcebe40a1a950f4f82a/users/227d1daba43c46da8bf2226d923cb87c/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:26.343380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fa9ab-45f5-4162-afdb-02205ec6555a service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:26.360718 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fa9ab-45f5-4162-afdb-02205ec6555a service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:26.477824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:26.536293 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24976382-d1f4-47fa-8c06-478836b6f31e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:26.538153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-24976382-d1f4-47fa-8c06-478836b6f31e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:26.548901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43596780-deab-419a-bed0-1ed5caec6469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:26.549634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43596780-deab-419a-bed0-1ed5caec6469 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:26.549934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43596780-deab-419a-bed0-1ed5caec6469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:26.550192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43596780-deab-419a-bed0-1ed5caec6469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:26.550944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43596780-deab-419a-bed0-1ed5caec6469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:26.552550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43596780-deab-419a-bed0-1ed5caec6469 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:26.597341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 882/1786] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:26.601382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43596780-deab-419a-bed0-1ed5caec6469 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:26.620498 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43596780-deab-419a-bed0-1ed5caec6469 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:26.629550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9dd6ed-6f33-45fd-b8b9-a4ed6d635f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:26.630463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dd6ed-6f33-45fd-b8b9-a4ed6d635f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:26.631008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dd6ed-6f33-45fd-b8b9-a4ed6d635f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:26.631448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dd6ed-6f33-45fd-b8b9-a4ed6d635f97 admin admin] PATH_INFO: `/v3/projects/0c6dd87228164dcebe40a1a950f4f82a/users/227d1daba43c46da8bf2226d923cb87c/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:26.636261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9dd6ed-6f33-45fd-b8b9-a4ed6d635f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6dd87228164dcebe40a1a950f4f82a, user_id=227d1daba43c46da8bf2226d923cb87c, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:26.643460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9dd6ed-6f33-45fd-b8b9-a4ed6d635f97 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:26.678194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 883/1787] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:26 2022] PUT /identity/v3/projects/0c6dd87228164dcebe40a1a950f4f82a/users/227d1daba43c46da8bf2226d923cb87c/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:26.682501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24976382-d1f4-47fa-8c06-478836b6f31e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:26.695143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65857968-5380-46a3-aa03-a84a42acffc9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:26.695595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65857968-5380-46a3-aa03-a84a42acffc9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:26.696128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65857968-5380-46a3-aa03-a84a42acffc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:26.785039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-65857968-5380-46a3-aa03-a84a42acffc9 None None] MFA Rules not processed for user `227d1daba43c46da8bf2226d923cb87c`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:26.796147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-65857968-5380-46a3-aa03-a84a42acffc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:26.847893 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5bfd4-c24b-4336-ba69-118beee4db74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:26.848663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5bfd4-c24b-4336-ba69-118beee4db74 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:26.849150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5bfd4-c24b-4336-ba69-118beee4db74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:26.849260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5bfd4-c24b-4336-ba69-118beee4db74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:26.855437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-84c5bfd4-c24b-4336-ba69-118beee4db74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:27.051290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5bfd4-c24b-4336-ba69-118beee4db74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:27.054488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5bfd4-c24b-4336-ba69-118beee4db74 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:27.084207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 905/1788] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:27.101861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 884/1789] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:27.157218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5bfd4-c24b-4336-ba69-118beee4db74 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:27.169866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5bfd4-c24b-4336-ba69-118beee4db74 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:27.176035 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeccd26-37c7-44fa-afb7-e38a13debfd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:27.176848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeccd26-37c7-44fa-afb7-e38a13debfd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:27.177352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeccd26-37c7-44fa-afb7-e38a13debfd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:27.177646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeccd26-37c7-44fa-afb7-e38a13debfd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:27.190311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeccd26-37c7-44fa-afb7-e38a13debfd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:27.191767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeccd26-37c7-44fa-afb7-e38a13debfd8 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:27.237399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 906/1790] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:28.364558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65857968-5380-46a3-aa03-a84a42acffc9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:28.368789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-65857968-5380-46a3-aa03-a84a42acffc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:28.463916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65857968-5380-46a3-aa03-a84a42acffc9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:28.475324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:28.475324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:28.475874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:28.476316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] PATH_INFO: `/v3/projects/5218b6e9c7664dc5b0716d6f551f9fab` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:28.477811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5218b6e9c7664dc5b0716d6f551f9fab)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:28.480550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:28.543230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5218b6e9c7664dc5b0716d6f551f9fab', 'request_id': 'req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:28.550762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5218b6e9c7664dc5b0716d6f551f9fab', 'request_id': 'req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:28.553248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:28.559507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5218b6e9c7664dc5b0716d6f551f9fab', 'request_id': 'req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:28.576942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] The token cache is being invalidate because project 5218b6e9c7664dc5b0716d6f551f9fab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:28.577173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:28.577871 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:28.580003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 885/1791] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:28 2022] DELETE /identity/v3/projects/5218b6e9c7664dc5b0716d6f551f9fab => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:28.583670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeccd26-37c7-44fa-afb7-e38a13debfd8 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:28.595778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeccd26-37c7-44fa-afb7-e38a13debfd8 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:28.665137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:28.682655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb72d3-18c2-4fe0-96bc-b9e9abe6a1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:28.683893 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb72d3-18c2-4fe0-96bc-b9e9abe6a1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:28.684374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb72d3-18c2-4fe0-96bc-b9e9abe6a1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:28.684804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb72d3-18c2-4fe0-96bc-b9e9abe6a1fc admin admin] PATH_INFO: `/v3/users/a5a482f5d0b3472faae96eb35bfc4db1` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:28.687777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb72d3-18c2-4fe0-96bc-b9e9abe6a1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5a482f5d0b3472faae96eb35bfc4db1)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:28.690423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb72d3-18c2-4fe0-96bc-b9e9abe6a1fc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:28.743985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c8eb72d3-18c2-4fe0-96bc-b9e9abe6a1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5a482f5d0b3472faae96eb35bfc4db1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:28.744686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c8eb72d3-18c2-4fe0-96bc-b9e9abe6a1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5a482f5d0b3472faae96eb35bfc4db1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:28.745297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c8eb72d3-18c2-4fe0-96bc-b9e9abe6a1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5a482f5d0b3472faae96eb35bfc4db1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:28.753584 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c8eb72d3-18c2-4fe0-96bc-b9e9abe6a1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5a482f5d0b3472faae96eb35bfc4db1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:28.761016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c8eb72d3-18c2-4fe0-96bc-b9e9abe6a1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5a482f5d0b3472faae96eb35bfc4db1'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:28.783852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 907/1792] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:28 2022] DELETE /identity/v3/users/a5a482f5d0b3472faae96eb35bfc4db1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:29.141117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:29.147965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:29.173919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ac162-e2f0-4ce3-9007-be9f6aac284d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:29.258989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:29.259595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.260436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.260718 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] PATH_INFO: `/v3/projects/314d41e4de2b48ffa65400e05a59854f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.261890 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=314d41e4de2b48ffa65400e05a59854f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:29.263800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:29.328794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '314d41e4de2b48ffa65400e05a59854f', 'request_id': 'req-6105bebf-7cb8-43a8-ab71-c68221cd2d05'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:29.337001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '314d41e4de2b48ffa65400e05a59854f', 'request_id': 'req-6105bebf-7cb8-43a8-ab71-c68221cd2d05'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:29.342810 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '314d41e4de2b48ffa65400e05a59854f', 'request_id': 'req-6105bebf-7cb8-43a8-ab71-c68221cd2d05'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:29.356719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] The token cache is being invalidate because project 314d41e4de2b48ffa65400e05a59854f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:29.357468 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:29.358255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6105bebf-7cb8-43a8-ab71-c68221cd2d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:29.363471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 886/1793] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:29 2022] DELETE /identity/v3/projects/314d41e4de2b48ffa65400e05a59854f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:29.365071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984bca1-b462-4336-a743-bc2d754cf8ba None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.365359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984bca1-b462-4336-a743-bc2d754cf8ba None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.366169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984bca1-b462-4336-a743-bc2d754cf8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.393806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1984bca1-b462-4336-a743-bc2d754cf8ba None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:29.395217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1984bca1-b462-4336-a743-bc2d754cf8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:29.487640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 908/1794] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:29.489148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9912e0c-0007-4a8d-a49a-5869ace30882 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.489620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9912e0c-0007-4a8d-a49a-5869ace30882 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.489952 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9912e0c-0007-4a8d-a49a-5869ace30882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.512260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b9912e0c-0007-4a8d-a49a-5869ace30882 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:29.513745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b9912e0c-0007-4a8d-a49a-5869ace30882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:29.526968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 887/1795] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:29.528298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984bca1-b462-4336-a743-bc2d754cf8ba None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:29.539168 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984bca1-b462-4336-a743-bc2d754cf8ba None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:29.544129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a370f-5e00-4eea-8154-5e2f9ab49874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:29.544835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a370f-5e00-4eea-8154-5e2f9ab49874 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.545372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a370f-5e00-4eea-8154-5e2f9ab49874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.545484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a370f-5e00-4eea-8154-5e2f9ab49874 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.546247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a370f-5e00-4eea-8154-5e2f9ab49874 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:29.546614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a370f-5e00-4eea-8154-5e2f9ab49874 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:29.549198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a370f-5e00-4eea-8154-5e2f9ab49874 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:29.564145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 909/1796] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:29.566588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9912e0c-0007-4a8d-a49a-5869ace30882 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:29.580248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9912e0c-0007-4a8d-a49a-5869ace30882 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:29.584954 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0c56d-2e84-483f-9191-dbb06c6f6417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:29.585551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0c56d-2e84-483f-9191-dbb06c6f6417 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.585831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0c56d-2e84-483f-9191-dbb06c6f6417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.586100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0c56d-2e84-483f-9191-dbb06c6f6417 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.586731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0c56d-2e84-483f-9191-dbb06c6f6417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:29.589296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0c56d-2e84-483f-9191-dbb06c6f6417 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:29.622130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 888/1797] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:29 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:29.623179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a370f-5e00-4eea-8154-5e2f9ab49874 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:29.633413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a370f-5e00-4eea-8154-5e2f9ab49874 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:29.636921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0c56d-2e84-483f-9191-dbb06c6f6417 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:29.638646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e3b0c56d-2e84-483f-9191-dbb06c6f6417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:29.685325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:29.687722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:29.692923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420974da-fd9c-4e50-90c9-858b530c8629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:29.693964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420974da-fd9c-4e50-90c9-858b530c8629 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.694292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420974da-fd9c-4e50-90c9-858b530c8629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.694761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420974da-fd9c-4e50-90c9-858b530c8629 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.695812 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420974da-fd9c-4e50-90c9-858b530c8629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:29.698213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420974da-fd9c-4e50-90c9-858b530c8629 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:29.707272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0c56d-2e84-483f-9191-dbb06c6f6417 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:29.712731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41c975-a79c-4f07-8742-e028abc15150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:29.713424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41c975-a79c-4f07-8742-e028abc15150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.713752 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41c975-a79c-4f07-8742-e028abc15150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.713977 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41c975-a79c-4f07-8742-e028abc15150 admin admin] PATH_INFO: `/v3/users/aea8f880ef934a96b51e2bdf1da92bc7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.715302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41c975-a79c-4f07-8742-e028abc15150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aea8f880ef934a96b51e2bdf1da92bc7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:29.717564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41c975-a79c-4f07-8742-e028abc15150 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:29.720713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 910/1798] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:29 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:29.724758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420974da-fd9c-4e50-90c9-858b530c8629 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:29.734171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420974da-fd9c-4e50-90c9-858b530c8629 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:29.739090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22fb85-e7a2-436c-9ebc-64984eb8d0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:29.739677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22fb85-e7a2-436c-9ebc-64984eb8d0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.739961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22fb85-e7a2-436c-9ebc-64984eb8d0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.740269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22fb85-e7a2-436c-9ebc-64984eb8d0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.740910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22fb85-e7a2-436c-9ebc-64984eb8d0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:29.742511 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22fb85-e7a2-436c-9ebc-64984eb8d0e7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:29.755732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 911/1799] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:29.758592 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22fb85-e7a2-436c-9ebc-64984eb8d0e7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:29.769893 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22fb85-e7a2-436c-9ebc-64984eb8d0e7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:29.776105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7b41c975-a79c-4f07-8742-e028abc15150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aea8f880ef934a96b51e2bdf1da92bc7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:29.776800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7b41c975-a79c-4f07-8742-e028abc15150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aea8f880ef934a96b51e2bdf1da92bc7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:29.776951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181fe49-624b-4160-a2ca-43fe1ce5be02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:29.777420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7b41c975-a79c-4f07-8742-e028abc15150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aea8f880ef934a96b51e2bdf1da92bc7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:29.777698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181fe49-624b-4160-a2ca-43fe1ce5be02 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.778037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181fe49-624b-4160-a2ca-43fe1ce5be02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.778380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181fe49-624b-4160-a2ca-43fe1ce5be02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.779099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181fe49-624b-4160-a2ca-43fe1ce5be02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:29.780935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181fe49-624b-4160-a2ca-43fe1ce5be02 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:29.791954 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7b41c975-a79c-4f07-8742-e028abc15150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aea8f880ef934a96b51e2bdf1da92bc7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:29.793179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 912/1800] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:29.797424 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181fe49-624b-4160-a2ca-43fe1ce5be02 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:29.799418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5181fe49-624b-4160-a2ca-43fe1ce5be02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:29.803063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7b41c975-a79c-4f07-8742-e028abc15150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aea8f880ef934a96b51e2bdf1da92bc7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:29.812198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181fe49-624b-4160-a2ca-43fe1ce5be02 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:29.818585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec50e3b-565a-4cb1-ac40-d16fdd8281f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:29.819236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec50e3b-565a-4cb1-ac40-d16fdd8281f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.819518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec50e3b-565a-4cb1-ac40-d16fdd8281f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.819797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec50e3b-565a-4cb1-ac40-d16fdd8281f8 admin admin] PATH_INFO: `/v3/projects/9c60c9be93d347d5aa86bc6f0806083a/users/b10c9d5876b64ad7b5e21b5728fd53ba/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.821515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 889/1801] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:29 2022] DELETE /identity/v3/users/aea8f880ef934a96b51e2bdf1da92bc7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:29.825895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86cdac-cd2e-4498-ad38-548a0ecebb18 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.826178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86cdac-cd2e-4498-ad38-548a0ecebb18 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.826451 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86cdac-cd2e-4498-ad38-548a0ecebb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.838939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec50e3b-565a-4cb1-ac40-d16fdd8281f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c60c9be93d347d5aa86bc6f0806083a, user_id=b10c9d5876b64ad7b5e21b5728fd53ba, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:29.841627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec50e3b-565a-4cb1-ac40-d16fdd8281f8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:29.860914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1a86cdac-cd2e-4498-ad38-548a0ecebb18 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:29.861225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 913/1802] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:29 2022] PUT /identity/v3/projects/9c60c9be93d347d5aa86bc6f0806083a/users/b10c9d5876b64ad7b5e21b5728fd53ba/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:29.868127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1a86cdac-cd2e-4498-ad38-548a0ecebb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:29.872512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a01095-46e3-4287-a66e-9e4b7e92b6b0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.872869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a01095-46e3-4287-a66e-9e4b7e92b6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.873160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a01095-46e3-4287-a66e-9e4b7e92b6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.901065 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e9a01095-46e3-4287-a66e-9e4b7e92b6b0 None None] MFA Rules not processed for user `b10c9d5876b64ad7b5e21b5728fd53ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:29.902271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e9a01095-46e3-4287-a66e-9e4b7e92b6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:29.924606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 890/1803] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:29.934242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86cdac-cd2e-4498-ad38-548a0ecebb18 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:29.935619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1a86cdac-cd2e-4498-ad38-548a0ecebb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:29.969366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 914/1804] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:29.973508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a01095-46e3-4287-a66e-9e4b7e92b6b0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:29.984319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a01095-46e3-4287-a66e-9e4b7e92b6b0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:29.989169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4206d-52ec-4678-8ad6-8da55c61e054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:29.989763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4206d-52ec-4678-8ad6-8da55c61e054 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.990052 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4206d-52ec-4678-8ad6-8da55c61e054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.990326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4206d-52ec-4678-8ad6-8da55c61e054 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:29.990951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4206d-52ec-4678-8ad6-8da55c61e054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:29.992937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4206d-52ec-4678-8ad6-8da55c61e054 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:29.993217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86cdac-cd2e-4498-ad38-548a0ecebb18 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:29.997244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d8a3a-beaa-4049-a1c8-439a6ac286c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:29.997852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d8a3a-beaa-4049-a1c8-439a6ac286c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:29.998271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d8a3a-beaa-4049-a1c8-439a6ac286c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:29.998562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d8a3a-beaa-4049-a1c8-439a6ac286c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.012117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d8a3a-beaa-4049-a1c8-439a6ac286c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:30.013674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d8a3a-beaa-4049-a1c8-439a6ac286c9 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:30.027643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 891/1805] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:30.033211 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 915/1806] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:29 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:30.037032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d8a3a-beaa-4049-a1c8-439a6ac286c9 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:30.049165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d8a3a-beaa-4049-a1c8-439a6ac286c9 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:30.054895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:30.122923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5662c7c-c05a-4527-98ed-339506ecd59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:30.123594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5662c7c-c05a-4527-98ed-339506ecd59e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:30.123882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5662c7c-c05a-4527-98ed-339506ecd59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:30.124203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5662c7c-c05a-4527-98ed-339506ecd59e admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.125348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5662c7c-c05a-4527-98ed-339506ecd59e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:30.127390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5662c7c-c05a-4527-98ed-339506ecd59e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:30.156683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 892/1807] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:30 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:30.161167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4206d-52ec-4678-8ad6-8da55c61e054 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:30.172170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4206d-52ec-4678-8ad6-8da55c61e054 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:30.177752 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b80396-01e4-4035-9023-d1d918439aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:30.178386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b80396-01e4-4035-9023-d1d918439aff admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:30.178670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b80396-01e4-4035-9023-d1d918439aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:30.178943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b80396-01e4-4035-9023-d1d918439aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.179595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b80396-01e4-4035-9023-d1d918439aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:30.181258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b80396-01e4-4035-9023-d1d918439aff admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:30.192664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 916/1808] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:30.198462 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5662c7c-c05a-4527-98ed-339506ecd59e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:30.211245 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5662c7c-c05a-4527-98ed-339506ecd59e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:30.216850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ffd225-a6fd-49d2-a7d6-d5b4375b8dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:30.217523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffd225-a6fd-49d2-a7d6-d5b4375b8dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:30.217805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffd225-a6fd-49d2-a7d6-d5b4375b8dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:30.218081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffd225-a6fd-49d2-a7d6-d5b4375b8dc3 admin admin] PATH_INFO: `/v3/projects/2449ae860c114dc59736150c87c4ac79/users/7f8bad2abd0b4799bdbd7efb8c903b22/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.236888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ffd225-a6fd-49d2-a7d6-d5b4375b8dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2449ae860c114dc59736150c87c4ac79, user_id=7f8bad2abd0b4799bdbd7efb8c903b22, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:30.239738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ffd225-a6fd-49d2-a7d6-d5b4375b8dc3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:30.259095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 893/1809] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:30 2022] PUT /identity/v3/projects/2449ae860c114dc59736150c87c4ac79/users/7f8bad2abd0b4799bdbd7efb8c903b22/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:30.260147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b80396-01e4-4035-9023-d1d918439aff admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:30.277744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b80396-01e4-4035-9023-d1d918439aff admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:30.347488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:30.351590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ffd225-a6fd-49d2-a7d6-d5b4375b8dc3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:30.353767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b7ffd225-a6fd-49d2-a7d6-d5b4375b8dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:30.367137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2c933-ce95-4d59-811e-de50a929dd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:30.367757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2c933-ce95-4d59-811e-de50a929dd60 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:30.368068 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2c933-ce95-4d59-811e-de50a929dd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:30.368332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2c933-ce95-4d59-811e-de50a929dd60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.369383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2c933-ce95-4d59-811e-de50a929dd60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:30.372440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2c933-ce95-4d59-811e-de50a929dd60 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:30.386956 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 917/1810] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:30.393541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2c933-ce95-4d59-811e-de50a929dd60 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:30.404629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2c933-ce95-4d59-811e-de50a929dd60 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:30.410338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467310e-3b75-4713-ab06-0a7614ca83aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:30.410998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467310e-3b75-4713-ab06-0a7614ca83aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:30.411274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467310e-3b75-4713-ab06-0a7614ca83aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:30.411546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467310e-3b75-4713-ab06-0a7614ca83aa admin admin] PATH_INFO: `/v3/projects/2449ae860c114dc59736150c87c4ac79/users/7f8bad2abd0b4799bdbd7efb8c903b22/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.413512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467310e-3b75-4713-ab06-0a7614ca83aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2449ae860c114dc59736150c87c4ac79, user_id=7f8bad2abd0b4799bdbd7efb8c903b22, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:30.416221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467310e-3b75-4713-ab06-0a7614ca83aa admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:30.430694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 918/1811] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:30 2022] PUT /identity/v3/projects/2449ae860c114dc59736150c87c4ac79/users/7f8bad2abd0b4799bdbd7efb8c903b22/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:30.441555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7241fb-d5f0-4c12-9042-b58aecdfb455 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:30.441847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7241fb-d5f0-4c12-9042-b58aecdfb455 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:30.442129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7241fb-d5f0-4c12-9042-b58aecdfb455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.456070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ffd225-a6fd-49d2-a7d6-d5b4375b8dc3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:30.478076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1a7241fb-d5f0-4c12-9042-b58aecdfb455 None None] MFA Rules not processed for user `7f8bad2abd0b4799bdbd7efb8c903b22`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:30.479479 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1a7241fb-d5f0-4c12-9042-b58aecdfb455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:30.523895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:30.524674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:30.524920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:30.525116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] PATH_INFO: `/v3/projects/13458a917c8044fbae06394f7d2b74e1` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.526282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13458a917c8044fbae06394f7d2b74e1)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:30.528249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:30.554403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:30.575923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 919/1812] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:30.584497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13458a917c8044fbae06394f7d2b74e1', 'request_id': 'req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:30.588753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7241fb-d5f0-4c12-9042-b58aecdfb455 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:30.590183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1a7241fb-d5f0-4c12-9042-b58aecdfb455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:30.591844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13458a917c8044fbae06394f7d2b74e1', 'request_id': 'req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:30.597484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13458a917c8044fbae06394f7d2b74e1', 'request_id': 'req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:30.612822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] The token cache is being invalidate because project 13458a917c8044fbae06394f7d2b74e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:30.613158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:30.616304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:30.618068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 894/1813] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:30 2022] DELETE /identity/v3/projects/13458a917c8044fbae06394f7d2b74e1 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:30.621658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:30.623115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:30.653104 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7241fb-d5f0-4c12-9042-b58aecdfb455 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:30.705788 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1fed6-9abe-4e82-8114-04e2daeb42e7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:30.707776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd226b-9118-4c64-8fef-a66024e7b91e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:30.708564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd226b-9118-4c64-8fef-a66024e7b91e service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:30.708915 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd226b-9118-4c64-8fef-a66024e7b91e service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:30.709200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd226b-9118-4c64-8fef-a66024e7b91e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.710977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a1cd226b-9118-4c64-8fef-a66024e7b91e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:30.711331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b485ba9-a698-47ce-bcc0-75002a69ba6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:30.712136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b485ba9-a698-47ce-bcc0-75002a69ba6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:30.712523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b485ba9-a698-47ce-bcc0-75002a69ba6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:30.712874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b485ba9-a698-47ce-bcc0-75002a69ba6b admin admin] PATH_INFO: `/v3/users/227d1daba43c46da8bf2226d923cb87c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.714250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b485ba9-a698-47ce-bcc0-75002a69ba6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=227d1daba43c46da8bf2226d923cb87c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:30.716346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b485ba9-a698-47ce-bcc0-75002a69ba6b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:30.752792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:30.779297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8b485ba9-a698-47ce-bcc0-75002a69ba6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '227d1daba43c46da8bf2226d923cb87c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:30.780092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8b485ba9-a698-47ce-bcc0-75002a69ba6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '227d1daba43c46da8bf2226d923cb87c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:30.780794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8b485ba9-a698-47ce-bcc0-75002a69ba6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '227d1daba43c46da8bf2226d923cb87c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:30.792781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd226b-9118-4c64-8fef-a66024e7b91e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:30.794371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8b485ba9-a698-47ce-bcc0-75002a69ba6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '227d1daba43c46da8bf2226d923cb87c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:30.794545 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd226b-9118-4c64-8fef-a66024e7b91e service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:30.804489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8b485ba9-a698-47ce-bcc0-75002a69ba6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '227d1daba43c46da8bf2226d923cb87c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:30.810261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 920/1814] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:30.834486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 895/1815] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:30 2022] DELETE /identity/v3/users/227d1daba43c46da8bf2226d923cb87c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:30.858532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd226b-9118-4c64-8fef-a66024e7b91e service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:30.861864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a1cd226b-9118-4c64-8fef-a66024e7b91e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:30.881813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd226b-9118-4c64-8fef-a66024e7b91e service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:30.989050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:30.990335 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:30.990839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:30.991375 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] PATH_INFO: `/v3/users/b10c9d5876b64ad7b5e21b5728fd53ba` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.993394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b10c9d5876b64ad7b5e21b5728fd53ba)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:30.996494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:31.108716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b10c9d5876b64ad7b5e21b5728fd53ba'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:31.110155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b10c9d5876b64ad7b5e21b5728fd53ba'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:31.111262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b10c9d5876b64ad7b5e21b5728fd53ba'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:31.121225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b10c9d5876b64ad7b5e21b5728fd53ba'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:31.130734 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b10c9d5876b64ad7b5e21b5728fd53ba'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:31.155110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 921/1816] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:30 2022] DELETE /identity/v3/users/b10c9d5876b64ad7b5e21b5728fd53ba => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:31.161120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b152-df6a-4c8d-916c-9a2e29815bbf None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:31.161621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b152-df6a-4c8d-916c-9a2e29815bbf None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:31.162006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b152-df6a-4c8d-916c-9a2e29815bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:31.188075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6d27b152-df6a-4c8d-916c-9a2e29815bbf None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:31.189466 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6d27b152-df6a-4c8d-916c-9a2e29815bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:31.227537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:31.268281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:31.270323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:31.313000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 896/1817] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:31.322235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27b152-df6a-4c8d-916c-9a2e29815bbf None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:31.330824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6d27b152-df6a-4c8d-916c-9a2e29815bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:31.464534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:31.528308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27b152-df6a-4c8d-916c-9a2e29815bbf None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:31.534340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95137d-0f2a-4762-88c8-6597480bfe3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:31.535115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95137d-0f2a-4762-88c8-6597480bfe3c service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:31.535393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95137d-0f2a-4762-88c8-6597480bfe3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:31.535731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95137d-0f2a-4762-88c8-6597480bfe3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:31.545040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f99e44-2ba3-45c1-9ef4-2559f30c76ec admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:31.564779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:31.565721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:31.566214 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:31.566699 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] PATH_INFO: `/v3/projects/0c6dd87228164dcebe40a1a950f4f82a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:31.568809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c6dd87228164dcebe40a1a950f4f82a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:31.570996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:31.573826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95137d-0f2a-4762-88c8-6597480bfe3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:31.575322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95137d-0f2a-4762-88c8-6597480bfe3c service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:31.599354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 897/1818] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:31.659514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c6dd87228164dcebe40a1a950f4f82a', 'request_id': 'req-861c660f-68f9-435b-8a4a-839c6764b9cb'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:31.668772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c6dd87228164dcebe40a1a950f4f82a', 'request_id': 'req-861c660f-68f9-435b-8a4a-839c6764b9cb'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:31.679145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c6dd87228164dcebe40a1a950f4f82a', 'request_id': 'req-861c660f-68f9-435b-8a4a-839c6764b9cb'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:31.724656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] The token cache is being invalidate because project 0c6dd87228164dcebe40a1a950f4f82a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:31.726355 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:31.727304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-861c660f-68f9-435b-8a4a-839c6764b9cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:31.730667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 922/1819] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:31 2022] DELETE /identity/v3/projects/0c6dd87228164dcebe40a1a950f4f82a => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:31.735244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d0af-637b-4f52-a293-7663c9c61f5b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:31.735537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d0af-637b-4f52-a293-7663c9c61f5b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:31.736086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d0af-637b-4f52-a293-7663c9c61f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:31.769936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-34d9d0af-637b-4f52-a293-7663c9c61f5b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:31.772090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-34d9d0af-637b-4f52-a293-7663c9c61f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:31.886893 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8323445-ab54-4c8c-a707-8b52c7988cef None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:31.887183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8323445-ab54-4c8c-a707-8b52c7988cef None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:31.887460 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8323445-ab54-4c8c-a707-8b52c7988cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:31.903438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 898/1820] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:31.940351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d8323445-ab54-4c8c-a707-8b52c7988cef None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:31.945160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d8323445-ab54-4c8c-a707-8b52c7988cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:31.966299 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 923/1821] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:31.971904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9d0af-637b-4f52-a293-7663c9c61f5b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:31.986736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9d0af-637b-4f52-a293-7663c9c61f5b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:32.003159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650d2d4-3730-4015-a2b8-d87003a6a2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:32.005651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650d2d4-3730-4015-a2b8-d87003a6a2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:32.005939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650d2d4-3730-4015-a2b8-d87003a6a2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:32.006207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650d2d4-3730-4015-a2b8-d87003a6a2c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:32.007003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650d2d4-3730-4015-a2b8-d87003a6a2c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:32.007306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650d2d4-3730-4015-a2b8-d87003a6a2c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:32.016529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650d2d4-3730-4015-a2b8-d87003a6a2c0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:32.046333 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8323445-ab54-4c8c-a707-8b52c7988cef None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:32.053753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 899/1822] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:32.059086 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8323445-ab54-4c8c-a707-8b52c7988cef None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:32.066387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d1834-c81b-4e03-9e15-f20b3d2b7fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:32.067312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d1834-c81b-4e03-9e15-f20b3d2b7fce admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:32.067600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d1834-c81b-4e03-9e15-f20b3d2b7fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:32.067875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d1834-c81b-4e03-9e15-f20b3d2b7fce admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:32.068701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d1834-c81b-4e03-9e15-f20b3d2b7fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:32.070761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d1834-c81b-4e03-9e15-f20b3d2b7fce admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:32.119413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 924/1823] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:32 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:32.123348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650d2d4-3730-4015-a2b8-d87003a6a2c0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:32.142853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650d2d4-3730-4015-a2b8-d87003a6a2c0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:32.346717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d5da8-f23e-474f-88e2-134b7a71d536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:32.348366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d5da8-f23e-474f-88e2-134b7a71d536 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:32.348974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d5da8-f23e-474f-88e2-134b7a71d536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:32.349455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d5da8-f23e-474f-88e2-134b7a71d536 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:32.351604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d5da8-f23e-474f-88e2-134b7a71d536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:32.360159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d5da8-f23e-474f-88e2-134b7a71d536 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:32.381041 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:32.419533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 900/1824] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:32 2022] POST /identity/v3/users => generated 294 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:32.421401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d1834-c81b-4e03-9e15-f20b3d2b7fce admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:32.438656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d1834-c81b-4e03-9e15-f20b3d2b7fce admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:32.445955 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bf51f-141e-4a2f-85b1-c062634cad28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:32.447048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf51f-141e-4a2f-85b1-c062634cad28 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:32.447623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf51f-141e-4a2f-85b1-c062634cad28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:32.447960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf51f-141e-4a2f-85b1-c062634cad28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:32.449049 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bf51f-141e-4a2f-85b1-c062634cad28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:32.451025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bf51f-141e-4a2f-85b1-c062634cad28 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:32.471568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d5da8-f23e-474f-88e2-134b7a71d536 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:32.483543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 925/1825] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:32.487101 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d5da8-f23e-474f-88e2-134b7a71d536 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:32.499725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47ee9f-8ed0-4942-90f8-8ab93ce9b75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:32.500465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47ee9f-8ed0-4942-90f8-8ab93ce9b75c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:32.500743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47ee9f-8ed0-4942-90f8-8ab93ce9b75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:32.501017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47ee9f-8ed0-4942-90f8-8ab93ce9b75c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:32.501714 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47ee9f-8ed0-4942-90f8-8ab93ce9b75c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:32.503432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47ee9f-8ed0-4942-90f8-8ab93ce9b75c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:32.528911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bf51f-141e-4a2f-85b1-c062634cad28 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:32.539474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 901/1826] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:32.543816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bf51f-141e-4a2f-85b1-c062634cad28 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:32.556170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23125c-bda4-4ce7-83d0-c8dd06afb6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:32.556846 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23125c-bda4-4ce7-83d0-c8dd06afb6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:32.557127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23125c-bda4-4ce7-83d0-c8dd06afb6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:32.557402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23125c-bda4-4ce7-83d0-c8dd06afb6c9 admin admin] PATH_INFO: `/v3/projects/5d6b0c7265db4b179baa18f7195ba632/users/dff03ba82ee24c71b97c7031636a1d82/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:32.584612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23125c-bda4-4ce7-83d0-c8dd06afb6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d6b0c7265db4b179baa18f7195ba632, user_id=dff03ba82ee24c71b97c7031636a1d82, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:32.587749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23125c-bda4-4ce7-83d0-c8dd06afb6c9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:32.665740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 926/1827] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:32 2022] PUT /identity/v3/projects/5d6b0c7265db4b179baa18f7195ba632/users/dff03ba82ee24c71b97c7031636a1d82/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:32.680533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6654615-69f2-4ade-8db5-2e7b854a0fad None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:32.680869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6654615-69f2-4ade-8db5-2e7b854a0fad None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:32.681235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6654615-69f2-4ade-8db5-2e7b854a0fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:32.781612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e6654615-69f2-4ade-8db5-2e7b854a0fad None None] MFA Rules not processed for user `dff03ba82ee24c71b97c7031636a1d82`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:32.794960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e6654615-69f2-4ade-8db5-2e7b854a0fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:32.893644 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23125c-bda4-4ce7-83d0-c8dd06afb6c9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:32.894433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 902/1828] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:33.035273 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23125c-bda4-4ce7-83d0-c8dd06afb6c9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:33.051413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecba0e-fdb6-4f99-a4c6-02d75bd27345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:33.051413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecba0e-fdb6-4f99-a4c6-02d75bd27345 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:33.051413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecba0e-fdb6-4f99-a4c6-02d75bd27345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:33.051788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecba0e-fdb6-4f99-a4c6-02d75bd27345 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:33.052329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecba0e-fdb6-4f99-a4c6-02d75bd27345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:33.054492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecba0e-fdb6-4f99-a4c6-02d75bd27345 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:33.125953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 927/1829] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:32 2022] POST /identity/v3/projects => generated 387 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:33.130633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6654615-69f2-4ade-8db5-2e7b854a0fad None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:33.149025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6654615-69f2-4ade-8db5-2e7b854a0fad None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:33.333891 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35276e02-7fdc-4e83-9fe5-b2b87bc3116e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:33.334507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35276e02-7fdc-4e83-9fe5-b2b87bc3116e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:33.334785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35276e02-7fdc-4e83-9fe5-b2b87bc3116e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:33.335155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35276e02-7fdc-4e83-9fe5-b2b87bc3116e admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:33.335809 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35276e02-7fdc-4e83-9fe5-b2b87bc3116e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:33.338078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35276e02-7fdc-4e83-9fe5-b2b87bc3116e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:33.390222 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecba0e-fdb6-4f99-a4c6-02d75bd27345 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:33.392666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-77ecba0e-fdb6-4f99-a4c6-02d75bd27345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:33.409693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:33.426625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecba0e-fdb6-4f99-a4c6-02d75bd27345 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:33.426881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 903/1830] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:33 2022] POST /identity/v3/users => generated 300 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:33.431109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35276e02-7fdc-4e83-9fe5-b2b87bc3116e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:33.441760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:33.442475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:33.442789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:33.443066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] PATH_INFO: `/v3/projects/9c60c9be93d347d5aa86bc6f0806083a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:33.444397 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c60c9be93d347d5aa86bc6f0806083a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:33.447302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:33.450621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35276e02-7fdc-4e83-9fe5-b2b87bc3116e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:33.464084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7730f-7e2f-4a33-95e5-11043dd7c28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:33.464696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7730f-7e2f-4a33-95e5-11043dd7c28f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:33.464982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7730f-7e2f-4a33-95e5-11043dd7c28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:33.465350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7730f-7e2f-4a33-95e5-11043dd7c28f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:33.466634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7730f-7e2f-4a33-95e5-11043dd7c28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:33.468394 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7730f-7e2f-4a33-95e5-11043dd7c28f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:33.493377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 904/1831] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:33.499067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7730f-7e2f-4a33-95e5-11043dd7c28f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:33.528227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7730f-7e2f-4a33-95e5-11043dd7c28f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:33.539131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a4b6d-ff0f-42f7-87c2-343230ed1dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:33.539997 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a4b6d-ff0f-42f7-87c2-343230ed1dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:33.541011 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a4b6d-ff0f-42f7-87c2-343230ed1dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:33.541011 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a4b6d-ff0f-42f7-87c2-343230ed1dc8 admin admin] PATH_INFO: `/v3/projects/b7a40b78ebda4f8bb85db7f421a7718f/users/4eef9364790c4bc78521ee5d7d6f2a93/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:33.571875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c60c9be93d347d5aa86bc6f0806083a', 'request_id': 'req-1478bb90-d196-4cd0-9744-ab60b024df25'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.576607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a4b6d-ff0f-42f7-87c2-343230ed1dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a40b78ebda4f8bb85db7f421a7718f, user_id=4eef9364790c4bc78521ee5d7d6f2a93, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:33.580411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a4b6d-ff0f-42f7-87c2-343230ed1dc8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:33.585221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c60c9be93d347d5aa86bc6f0806083a', 'request_id': 'req-1478bb90-d196-4cd0-9744-ab60b024df25'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.609708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 905/1832] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:33 2022] PUT /identity/v3/projects/b7a40b78ebda4f8bb85db7f421a7718f/users/4eef9364790c4bc78521ee5d7d6f2a93/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:33.611373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a4b6d-ff0f-42f7-87c2-343230ed1dc8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:33.613758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c60c9be93d347d5aa86bc6f0806083a', 'request_id': 'req-1478bb90-d196-4cd0-9744-ab60b024df25'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.645704 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] The token cache is being invalidate because project 9c60c9be93d347d5aa86bc6f0806083a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:33.646703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.647744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.650285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 928/1833] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:33 2022] DELETE /identity/v3/projects/9c60c9be93d347d5aa86bc6f0806083a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:33.655272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:33.662545 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:33.706015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:33.822264 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a4b6d-ff0f-42f7-87c2-343230ed1dc8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:33.837697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e3e22-7977-4d86-b357-eca8564e341a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:33.838224 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478bb90-d196-4cd0-9744-ab60b024df25 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:33.839009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e3e22-7977-4d86-b357-eca8564e341a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:33.839501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e3e22-7977-4d86-b357-eca8564e341a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:33.839922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e3e22-7977-4d86-b357-eca8564e341a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:33.840887 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e3e22-7977-4d86-b357-eca8564e341a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:33.842616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e3e22-7977-4d86-b357-eca8564e341a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:33.847150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:33.848790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:33.849306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:33.849663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] PATH_INFO: `/v3/users/7f8bad2abd0b4799bdbd7efb8c903b22` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:33.851131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8bad2abd0b4799bdbd7efb8c903b22)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:33.854126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:33.879223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 906/1834] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:33.892955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e3e22-7977-4d86-b357-eca8564e341a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:33.897242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ce6e3e22-7977-4d86-b357-eca8564e341a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:33.930505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e3e22-7977-4d86-b357-eca8564e341a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:33.941127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be7572-a2ea-45fc-81f7-f082af9d7cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:33.941762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be7572-a2ea-45fc-81f7-f082af9d7cac admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:33.942053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be7572-a2ea-45fc-81f7-f082af9d7cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:33.942332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be7572-a2ea-45fc-81f7-f082af9d7cac admin admin] PATH_INFO: `/v3/projects/b7a40b78ebda4f8bb85db7f421a7718f/users/4eef9364790c4bc78521ee5d7d6f2a93/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:33.953925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8bad2abd0b4799bdbd7efb8c903b22'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.954258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be7572-a2ea-45fc-81f7-f082af9d7cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a40b78ebda4f8bb85db7f421a7718f, user_id=4eef9364790c4bc78521ee5d7d6f2a93, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:33.955700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8bad2abd0b4799bdbd7efb8c903b22'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.957314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8bad2abd0b4799bdbd7efb8c903b22'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.958403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be7572-a2ea-45fc-81f7-f082af9d7cac admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:33.975764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8bad2abd0b4799bdbd7efb8c903b22'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.987774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8bad2abd0b4799bdbd7efb8c903b22'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.988571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 907/1835] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:33 2022] PUT /identity/v3/projects/b7a40b78ebda4f8bb85db7f421a7718f/users/4eef9364790c4bc78521ee5d7d6f2a93/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:34.017944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad9e33-6602-4e14-a85d-08f1d57b1d21 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:34.018740 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad9e33-6602-4e14-a85d-08f1d57b1d21 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:34.022395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad9e33-6602-4e14-a85d-08f1d57b1d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:34.040612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 929/1836] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:33 2022] DELETE /identity/v3/users/7f8bad2abd0b4799bdbd7efb8c903b22 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:34.076443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2bad9e33-6602-4e14-a85d-08f1d57b1d21 None None] MFA Rules not processed for user `4eef9364790c4bc78521ee5d7d6f2a93`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:34.078671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2bad9e33-6602-4e14-a85d-08f1d57b1d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:34.252173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 908/1837] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:34.272309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:34.274139 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:34.282096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:34.370197 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c51915-84b4-430f-944f-a94ecbae38b1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:34.376930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c941f-de59-43f9-9e49-4b3126069d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:34.377730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c941f-de59-43f9-9e49-4b3126069d3f service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:34.378152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c941f-de59-43f9-9e49-4b3126069d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:34.378605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c941f-de59-43f9-9e49-4b3126069d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:34.390210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c941f-de59-43f9-9e49-4b3126069d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:34.392350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c941f-de59-43f9-9e49-4b3126069d3f service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:34.405129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 930/1838] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:34.517805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad9e33-6602-4e14-a85d-08f1d57b1d21 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:34.520461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2bad9e33-6602-4e14-a85d-08f1d57b1d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:34.680832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad9e33-6602-4e14-a85d-08f1d57b1d21 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:34.690787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52f0a9-8158-4aa3-8ba2-1cd498085cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:34.691595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f0a9-8158-4aa3-8ba2-1cd498085cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:34.692207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f0a9-8158-4aa3-8ba2-1cd498085cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:34.693094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f0a9-8158-4aa3-8ba2-1cd498085cec admin admin] PATH_INFO: `/v3/users/dff03ba82ee24c71b97c7031636a1d82` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:34.694044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52f0a9-8158-4aa3-8ba2-1cd498085cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=dff03ba82ee24c71b97c7031636a1d82)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:34.696975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52f0a9-8158-4aa3-8ba2-1cd498085cec admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:34.758770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:34.776904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-da52f0a9-8158-4aa3-8ba2-1cd498085cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dff03ba82ee24c71b97c7031636a1d82'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:34.778052 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-da52f0a9-8158-4aa3-8ba2-1cd498085cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dff03ba82ee24c71b97c7031636a1d82'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:34.778904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-da52f0a9-8158-4aa3-8ba2-1cd498085cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dff03ba82ee24c71b97c7031636a1d82'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:34.794016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-da52f0a9-8158-4aa3-8ba2-1cd498085cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dff03ba82ee24c71b97c7031636a1d82'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:34.802295 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-da52f0a9-8158-4aa3-8ba2-1cd498085cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dff03ba82ee24c71b97c7031636a1d82'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:34.822640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c941f-de59-43f9-9e49-4b3126069d3f service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:34.824340 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b52c941f-de59-43f9-9e49-4b3126069d3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:34.840286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c941f-de59-43f9-9e49-4b3126069d3f service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:34.842709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 909/1839] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:34 2022] DELETE /identity/v3/users/dff03ba82ee24c71b97c7031636a1d82 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:34.847257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e91dc-3add-4795-b3d2-b8508875721d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:34.847737 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e91dc-3add-4795-b3d2-b8508875721d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:34.848279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e91dc-3add-4795-b3d2-b8508875721d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:34.871735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3b5e91dc-3add-4795-b3d2-b8508875721d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:34.873276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3b5e91dc-3add-4795-b3d2-b8508875721d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:34.938152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:34.938817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:34.939376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:34.940282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] PATH_INFO: `/v3/projects/2449ae860c114dc59736150c87c4ac79` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:34.942095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2449ae860c114dc59736150c87c4ac79)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:34.944690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:34.952317 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 910/1840] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:34.961532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e91dc-3add-4795-b3d2-b8508875721d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:34.962519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3b5e91dc-3add-4795-b3d2-b8508875721d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.007339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2449ae860c114dc59736150c87c4ac79', 'request_id': 'req-ea30358f-fd30-466e-8ff9-a166bd63c415'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.014573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2449ae860c114dc59736150c87c4ac79', 'request_id': 'req-ea30358f-fd30-466e-8ff9-a166bd63c415'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.023593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2449ae860c114dc59736150c87c4ac79', 'request_id': 'req-ea30358f-fd30-466e-8ff9-a166bd63c415'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.057402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e91dc-3add-4795-b3d2-b8508875721d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.059498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] The token cache is being invalidate because project 2449ae860c114dc59736150c87c4ac79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:35.060264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.061074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ea30358f-fd30-466e-8ff9-a166bd63c415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.064812 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 931/1841] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:34 2022] DELETE /identity/v3/projects/2449ae860c114dc59736150c87c4ac79 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:35.070078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486091f-1a2b-4775-91c8-3b0444edf4cb None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.070368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486091f-1a2b-4775-91c8-3b0444edf4cb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.070640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486091f-1a2b-4775-91c8-3b0444edf4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.096550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3486091f-1a2b-4775-91c8-3b0444edf4cb None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:35.098462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3486091f-1a2b-4775-91c8-3b0444edf4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.126192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c47fd-2f9f-41ef-b2f8-46c5738e5e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.128276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c47fd-2f9f-41ef-b2f8-46c5738e5e54 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.128276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c47fd-2f9f-41ef-b2f8-46c5738e5e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.128276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c47fd-2f9f-41ef-b2f8-46c5738e5e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.130074 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-661c47fd-2f9f-41ef-b2f8-46c5738e5e54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.174825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:35.182009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:35.235001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c47fd-2f9f-41ef-b2f8-46c5738e5e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.236747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c47fd-2f9f-41ef-b2f8-46c5738e5e54 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.250572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 932/1842] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:35.252205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865be1e-9749-47dc-8c86-68daf62bc08d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.252500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865be1e-9749-47dc-8c86-68daf62bc08d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.252777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865be1e-9749-47dc-8c86-68daf62bc08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.264108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 911/1843] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:35.283637 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6865be1e-9749-47dc-8c86-68daf62bc08d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:35.285389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6865be1e-9749-47dc-8c86-68daf62bc08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.297978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 933/1844] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:35.301506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c47fd-2f9f-41ef-b2f8-46c5738e5e54 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.323218 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c47fd-2f9f-41ef-b2f8-46c5738e5e54 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.330266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da9c33-6681-4f4d-9fc1-aa75bfe2c951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.334771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da9c33-6681-4f4d-9fc1-aa75bfe2c951 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.335558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da9c33-6681-4f4d-9fc1-aa75bfe2c951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.335905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da9c33-6681-4f4d-9fc1-aa75bfe2c951 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.336693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da9c33-6681-4f4d-9fc1-aa75bfe2c951 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:35.337002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da9c33-6681-4f4d-9fc1-aa75bfe2c951 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.338852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da9c33-6681-4f4d-9fc1-aa75bfe2c951 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.357171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 912/1845] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:35.362285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865be1e-9749-47dc-8c86-68daf62bc08d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.376582 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865be1e-9749-47dc-8c86-68daf62bc08d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.382956 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2614dd9-9588-4a4d-aa48-a234d25b16f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.383268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2614dd9-9588-4a4d-aa48-a234d25b16f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.383661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2614dd9-9588-4a4d-aa48-a234d25b16f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.383892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2614dd9-9588-4a4d-aa48-a234d25b16f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.385267 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2614dd9-9588-4a4d-aa48-a234d25b16f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.387597 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2614dd9-9588-4a4d-aa48-a234d25b16f9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.441254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 934/1846] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:35 2022] POST /identity/v3/projects => generated 393 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:35.442986 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da9c33-6681-4f4d-9fc1-aa75bfe2c951 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.460021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da9c33-6681-4f4d-9fc1-aa75bfe2c951 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.597919 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2614dd9-9588-4a4d-aa48-a234d25b16f9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.599850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b2614dd9-9588-4a4d-aa48-a234d25b16f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.628510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe52795-cbf7-44b1-8b54-cd21509c2730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.629139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52795-cbf7-44b1-8b54-cd21509c2730 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.629424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52795-cbf7-44b1-8b54-cd21509c2730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.631262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52795-cbf7-44b1-8b54-cd21509c2730 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.631262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe52795-cbf7-44b1-8b54-cd21509c2730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.632504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe52795-cbf7-44b1-8b54-cd21509c2730 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.672246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 913/1847] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:35 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:35.673612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe52795-cbf7-44b1-8b54-cd21509c2730 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.686032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe52795-cbf7-44b1-8b54-cd21509c2730 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.691750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed566064-d328-4fd3-a5c9-c35360ebf22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.692564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed566064-d328-4fd3-a5c9-c35360ebf22d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.692845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed566064-d328-4fd3-a5c9-c35360ebf22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.693267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed566064-d328-4fd3-a5c9-c35360ebf22d admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.693492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2614dd9-9588-4a4d-aa48-a234d25b16f9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.693920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed566064-d328-4fd3-a5c9-c35360ebf22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.695657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed566064-d328-4fd3-a5c9-c35360ebf22d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.699412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.700209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.700492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.700765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] PATH_INFO: `/v3/users/ff39a263247a45f8a2fe175aa63a9fd2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.701947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff39a263247a45f8a2fe175aa63a9fd2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.705397 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.708858 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 914/1848] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:35.712729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed566064-d328-4fd3-a5c9-c35360ebf22d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.727156 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed566064-d328-4fd3-a5c9-c35360ebf22d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.743181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8951a-1e24-4a1f-9416-42ba0d9bd9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.743883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8951a-1e24-4a1f-9416-42ba0d9bd9bb admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.744186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8951a-1e24-4a1f-9416-42ba0d9bd9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.744467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8951a-1e24-4a1f-9416-42ba0d9bd9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.745136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8951a-1e24-4a1f-9416-42ba0d9bd9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.746931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8951a-1e24-4a1f-9416-42ba0d9bd9bb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.772515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 915/1849] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:35.773469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8951a-1e24-4a1f-9416-42ba0d9bd9bb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.776148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-09e8951a-1e24-4a1f-9416-42ba0d9bd9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.791298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8951a-1e24-4a1f-9416-42ba0d9bd9bb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.794216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff39a263247a45f8a2fe175aa63a9fd2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.795094 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff39a263247a45f8a2fe175aa63a9fd2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.796221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff39a263247a45f8a2fe175aa63a9fd2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.796838 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.797573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.798296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.798296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] PATH_INFO: `/v3/projects/5d6b0c7265db4b179baa18f7195ba632` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.799531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d6b0c7265db4b179baa18f7195ba632)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.801735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.806608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff39a263247a45f8a2fe175aa63a9fd2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.815909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff39a263247a45f8a2fe175aa63a9fd2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.844076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 935/1850] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:35 2022] DELETE /identity/v3/users/ff39a263247a45f8a2fe175aa63a9fd2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:35.847462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.847462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.864155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d6b0c7265db4b179baa18f7195ba632', 'request_id': 'req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.871403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d6b0c7265db4b179baa18f7195ba632', 'request_id': 'req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.878200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d6b0c7265db4b179baa18f7195ba632', 'request_id': 'req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.882172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:35.885293 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:35.898647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] The token cache is being invalidate because project 5d6b0c7265db4b179baa18f7195ba632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:35.899215 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.900425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9b064e5-1a0d-4478-96c0-f9c6c0c9ff92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.902539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 916/1851] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:35 2022] DELETE /identity/v3/projects/5d6b0c7265db4b179baa18f7195ba632 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:35.905605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae9a9a-3f89-43cc-9f5c-ca71d7d33721 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.905605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae9a9a-3f89-43cc-9f5c-ca71d7d33721 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.905605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae9a9a-3f89-43cc-9f5c-ca71d7d33721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.927991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7eae9a9a-3f89-43cc-9f5c-ca71d7d33721 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:35.929240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7eae9a9a-3f89-43cc-9f5c-ca71d7d33721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.934777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354f67c-1213-4d0d-b8e3-b24aa5039ffe admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.997001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 917/1852] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:35.997938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae9a9a-3f89-43cc-9f5c-ca71d7d33721 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.999189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97cd40-6340-4e7b-9879-f113cde56c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.999313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7eae9a9a-3f89-43cc-9f5c-ca71d7d33721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.000404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97cd40-6340-4e7b-9879-f113cde56c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.006347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97cd40-6340-4e7b-9879-f113cde56c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.006643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97cd40-6340-4e7b-9879-f113cde56c50 admin admin] PATH_INFO: `/v3/projects/9f8ad395e42a42598af71dfcb251d76f/users/212ad17a91dc4a60a5c5ed1f05e0e123/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.012875 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae9a9a-3f89-43cc-9f5c-ca71d7d33721 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.018375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a6b21-a43c-45a0-93c4-2317c8ce1705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.019000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a6b21-a43c-45a0-93c4-2317c8ce1705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.019283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a6b21-a43c-45a0-93c4-2317c8ce1705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.019560 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a6b21-a43c-45a0-93c4-2317c8ce1705 admin admin] PATH_INFO: `/v3/users/4eef9364790c4bc78521ee5d7d6f2a93` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.020739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a6b21-a43c-45a0-93c4-2317c8ce1705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eef9364790c4bc78521ee5d7d6f2a93)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.022677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a6b21-a43c-45a0-93c4-2317c8ce1705 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.022783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97cd40-6340-4e7b-9879-f113cde56c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8ad395e42a42598af71dfcb251d76f, user_id=212ad17a91dc4a60a5c5ed1f05e0e123, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.025367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97cd40-6340-4e7b-9879-f113cde56c50 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.039084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 936/1853] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:35 2022] PUT /identity/v3/projects/9f8ad395e42a42598af71dfcb251d76f/users/212ad17a91dc4a60a5c5ed1f05e0e123/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:36.049306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97cd40-6340-4e7b-9879-f113cde56c50 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.049306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4a97cd40-6340-4e7b-9879-f113cde56c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.073296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-462a6b21-a43c-45a0-93c4-2317c8ce1705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eef9364790c4bc78521ee5d7d6f2a93'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.073965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-462a6b21-a43c-45a0-93c4-2317c8ce1705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eef9364790c4bc78521ee5d7d6f2a93'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.074607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-462a6b21-a43c-45a0-93c4-2317c8ce1705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eef9364790c4bc78521ee5d7d6f2a93'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.084670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-462a6b21-a43c-45a0-93c4-2317c8ce1705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eef9364790c4bc78521ee5d7d6f2a93'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.093104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-462a6b21-a43c-45a0-93c4-2317c8ce1705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eef9364790c4bc78521ee5d7d6f2a93'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.109603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97cd40-6340-4e7b-9879-f113cde56c50 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.112504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 918/1854] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:35 2022] DELETE /identity/v3/users/4eef9364790c4bc78521ee5d7d6f2a93 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:36.113542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5597a-1329-42c0-9ebf-ae450d2a80ba None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.113829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5597a-1329-42c0-9ebf-ae450d2a80ba None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.114103 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5597a-1329-42c0-9ebf-ae450d2a80ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.118288 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21110d48-dd99-4e2e-b7d6-58db1cf6be15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.120852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21110d48-dd99-4e2e-b7d6-58db1cf6be15 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.121541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21110d48-dd99-4e2e-b7d6-58db1cf6be15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.121955 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21110d48-dd99-4e2e-b7d6-58db1cf6be15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.133340 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-21110d48-dd99-4e2e-b7d6-58db1cf6be15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.152342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d5e5597a-1329-42c0-9ebf-ae450d2a80ba None None] MFA Rules not processed for user `212ad17a91dc4a60a5c5ed1f05e0e123`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:36.153654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d5e5597a-1329-42c0-9ebf-ae450d2a80ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.247093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21110d48-dd99-4e2e-b7d6-58db1cf6be15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.248649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21110d48-dd99-4e2e-b7d6-58db1cf6be15 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.263716 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:36.265220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 937/1855] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:36.283901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 919/1856] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:36.289036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21110d48-dd99-4e2e-b7d6-58db1cf6be15 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.291163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-21110d48-dd99-4e2e-b7d6-58db1cf6be15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.310533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21110d48-dd99-4e2e-b7d6-58db1cf6be15 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.319768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:36.404896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40dc03b-7f06-4b0b-b6a9-4eabe09d86cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.406377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dc03b-7f06-4b0b-b6a9-4eabe09d86cf admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.407629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dc03b-7f06-4b0b-b6a9-4eabe09d86cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.409012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dc03b-7f06-4b0b-b6a9-4eabe09d86cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.410611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40dc03b-7f06-4b0b-b6a9-4eabe09d86cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.413753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40dc03b-7f06-4b0b-b6a9-4eabe09d86cf admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.456515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 938/1857] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:36.468337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5597a-1329-42c0-9ebf-ae450d2a80ba None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.480212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5597a-1329-42c0-9ebf-ae450d2a80ba None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.561647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddce57-c4f2-4046-a669-b72ac8a9d18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.561647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddce57-c4f2-4046-a669-b72ac8a9d18a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.562619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddce57-c4f2-4046-a669-b72ac8a9d18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.563165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddce57-c4f2-4046-a669-b72ac8a9d18a admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.563824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddce57-c4f2-4046-a669-b72ac8a9d18a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.565995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddce57-c4f2-4046-a669-b72ac8a9d18a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.595255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 920/1858] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/users => generated 302 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:36.596069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40dc03b-7f06-4b0b-b6a9-4eabe09d86cf admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.607699 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40dc03b-7f06-4b0b-b6a9-4eabe09d86cf admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.614345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d8d0-fa46-4d4f-8b9e-dbf16e72d21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.615079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d8d0-fa46-4d4f-8b9e-dbf16e72d21f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.615437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d8d0-fa46-4d4f-8b9e-dbf16e72d21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.615753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d8d0-fa46-4d4f-8b9e-dbf16e72d21f admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.616517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98d8d0-fa46-4d4f-8b9e-dbf16e72d21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.619359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98d8d0-fa46-4d4f-8b9e-dbf16e72d21f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.635938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 939/1859] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:36.637401 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddce57-c4f2-4046-a669-b72ac8a9d18a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.652247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddce57-c4f2-4046-a669-b72ac8a9d18a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.657587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c58f8-5f91-49db-8f34-14eeb0337d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.666777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c58f8-5f91-49db-8f34-14eeb0337d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.666777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c58f8-5f91-49db-8f34-14eeb0337d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.666777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c58f8-5f91-49db-8f34-14eeb0337d75 admin admin] PATH_INFO: `/v3/projects/b8ff204c4e7941bf96c14d7db06877d0/users/1ec901d609804e9fa8be4bbab3b03d60/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.698233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d8d0-fa46-4d4f-8b9e-dbf16e72d21f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.702997 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cd98d8d0-fa46-4d4f-8b9e-dbf16e72d21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.729874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d8d0-fa46-4d4f-8b9e-dbf16e72d21f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.735051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c58f8-5f91-49db-8f34-14eeb0337d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ff204c4e7941bf96c14d7db06877d0, user_id=1ec901d609804e9fa8be4bbab3b03d60, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.739567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.740269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.740540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.743444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c58f8-5f91-49db-8f34-14eeb0337d75 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.743755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] PATH_INFO: `/v3/users/48f76882fbf8462d976d5e5c5cbd4571` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.745044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f76882fbf8462d976d5e5c5cbd4571)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.747260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.796358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 921/1860] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:36 2022] PUT /identity/v3/projects/b8ff204c4e7941bf96c14d7db06877d0/users/1ec901d609804e9fa8be4bbab3b03d60/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:36.797179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c58f8-5f91-49db-8f34-14eeb0337d75 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.799833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-081c58f8-5f91-49db-8f34-14eeb0337d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.874582 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f76882fbf8462d976d5e5c5cbd4571'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.875470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f76882fbf8462d976d5e5c5cbd4571'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.876837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f76882fbf8462d976d5e5c5cbd4571'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.887352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f76882fbf8462d976d5e5c5cbd4571'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.897827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f76882fbf8462d976d5e5c5cbd4571'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.926417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 940/1861] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:36 2022] DELETE /identity/v3/users/48f76882fbf8462d976d5e5c5cbd4571 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:36.927349 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.927740 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c58f8-5f91-49db-8f34-14eeb0337d75 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.929322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.019346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.020278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.020717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.020997 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] PATH_INFO: `/v3/projects/b7a40b78ebda4f8bb85db7f421a7718f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.022147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7a40b78ebda4f8bb85db7f421a7718f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.024177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.024289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:37.037661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:37.040419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da911e-9d7a-4fb0-b485-d6b3a570a353 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.054158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.055401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.055911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.056349 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] PATH_INFO: `/v3/users/7d75e0f4dbc940a7aa394a3ab60f5b82` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.058015 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d75e0f4dbc940a7aa394a3ab60f5b82)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.060736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.081503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7a40b78ebda4f8bb85db7f421a7718f', 'request_id': 'req-b7d17d47-072e-48d2-8c00-47708e907701'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.089613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7a40b78ebda4f8bb85db7f421a7718f', 'request_id': 'req-b7d17d47-072e-48d2-8c00-47708e907701'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.109755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7a40b78ebda4f8bb85db7f421a7718f', 'request_id': 'req-b7d17d47-072e-48d2-8c00-47708e907701'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.125283 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d75e0f4dbc940a7aa394a3ab60f5b82'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.126287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d75e0f4dbc940a7aa394a3ab60f5b82'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.127731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d75e0f4dbc940a7aa394a3ab60f5b82'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.131588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] The token cache is being invalidate because project b7a40b78ebda4f8bb85db7f421a7718f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:37.131588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.133311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.138083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 922/1862] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:36 2022] DELETE /identity/v3/projects/b7a40b78ebda4f8bb85db7f421a7718f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:37.139549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.142487 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d75e0f4dbc940a7aa394a3ab60f5b82'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.143742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.153159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d75e0f4dbc940a7aa394a3ab60f5b82'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.187973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 941/1863] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:36 2022] DELETE /identity/v3/users/7d75e0f4dbc940a7aa394a3ab60f5b82 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:37.188884 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.190390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.222361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d17d47-072e-48d2-8c00-47708e907701 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.310654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03d5ab-4003-4ec5-906d-3e43aec2ac98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.311464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03d5ab-4003-4ec5-906d-3e43aec2ac98 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.311748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03d5ab-4003-4ec5-906d-3e43aec2ac98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.312075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03d5ab-4003-4ec5-906d-3e43aec2ac98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.313005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03d5ab-4003-4ec5-906d-3e43aec2ac98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.314405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda8424-bea2-4e99-a5ff-8ce7694e9603 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.315036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03d5ab-4003-4ec5-906d-3e43aec2ac98 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.321323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.322323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.322798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.324481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] PATH_INFO: `/v3/projects/53f4489793504fba95d11f1e3b930842` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.326329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f4489793504fba95d11f1e3b930842)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.329272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.333657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 923/1864] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:37.333657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d41a4-0d63-4cbf-ba0b-b7dc42abf4d2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.333657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d41a4-0d63-4cbf-ba0b-b7dc42abf4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.333657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d41a4-0d63-4cbf-ba0b-b7dc42abf4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.355770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c03d41a4-0d63-4cbf-ba0b-b7dc42abf4d2 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:37.357046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c03d41a4-0d63-4cbf-ba0b-b7dc42abf4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.366575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 924/1865] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:37.367741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c758614-3eeb-4be8-8103-ac350467c6ac None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.368097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c758614-3eeb-4be8-8103-ac350467c6ac None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.368392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c758614-3eeb-4be8-8103-ac350467c6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.392505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5c758614-3eeb-4be8-8103-ac350467c6ac None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:37.394387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5c758614-3eeb-4be8-8103-ac350467c6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.396398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f4489793504fba95d11f1e3b930842', 'request_id': 'req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.408620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f4489793504fba95d11f1e3b930842', 'request_id': 'req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.417557 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f4489793504fba95d11f1e3b930842', 'request_id': 'req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.439038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] The token cache is being invalidate because project 53f4489793504fba95d11f1e3b930842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:37.439382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.440127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e50b17cc-2671-4b6a-8c74-1e15cc74afdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.442134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 942/1866] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:37 2022] DELETE /identity/v3/projects/53f4489793504fba95d11f1e3b930842 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:37.443732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d509057-f0ed-4cbb-8d2d-2ce6f1e9329d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.444074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d509057-f0ed-4cbb-8d2d-2ce6f1e9329d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.444354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d509057-f0ed-4cbb-8d2d-2ce6f1e9329d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.447317 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:37.460188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:37.473109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0d509057-f0ed-4cbb-8d2d-2ce6f1e9329d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:37.474522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0d509057-f0ed-4cbb-8d2d-2ce6f1e9329d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.501420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 925/1867] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:37.502130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c758614-3eeb-4be8-8103-ac350467c6ac None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.503647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5c758614-3eeb-4be8-8103-ac350467c6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.521426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c758614-3eeb-4be8-8103-ac350467c6ac None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.529901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dd034-47da-44d4-ac0f-b740064bac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.530837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd034-47da-44d4-ac0f-b740064bac79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.531197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd034-47da-44d4-ac0f-b740064bac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.531896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd034-47da-44d4-ac0f-b740064bac79 admin admin] PATH_INFO: `/v3/projects/b8ff204c4e7941bf96c14d7db06877d0/users/1ec901d609804e9fa8be4bbab3b03d60/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.535851 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8dd034-47da-44d4-ac0f-b740064bac79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ff204c4e7941bf96c14d7db06877d0, user_id=1ec901d609804e9fa8be4bbab3b03d60, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.539406 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8dd034-47da-44d4-ac0f-b740064bac79 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.543872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 943/1868] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:37.544933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d509057-f0ed-4cbb-8d2d-2ce6f1e9329d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.548210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0d509057-f0ed-4cbb-8d2d-2ce6f1e9329d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.561749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 926/1869] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:37 2022] PUT /identity/v3/projects/b8ff204c4e7941bf96c14d7db06877d0/users/1ec901d609804e9fa8be4bbab3b03d60/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:37.562804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dd034-47da-44d4-ac0f-b740064bac79 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.564508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7f8dd034-47da-44d4-ac0f-b740064bac79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.643868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d509057-f0ed-4cbb-8d2d-2ce6f1e9329d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.659989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dd034-47da-44d4-ac0f-b740064bac79 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.665670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cf66f-d55e-4779-9104-5f81f01b0910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.666418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cf66f-d55e-4779-9104-5f81f01b0910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.666698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cf66f-d55e-4779-9104-5f81f01b0910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.667086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cf66f-d55e-4779-9104-5f81f01b0910 admin admin] PATH_INFO: `/v3/users/ffe0758ed24d4832a544df58efc3fa35` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.668497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cf66f-d55e-4779-9104-5f81f01b0910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffe0758ed24d4832a544df58efc3fa35)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.670691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cf66f-d55e-4779-9104-5f81f01b0910 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.719538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06372d-8a77-40a9-8086-9e85698b53a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.720810 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06372d-8a77-40a9-8086-9e85698b53a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.720810 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06372d-8a77-40a9-8086-9e85698b53a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.721184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06372d-8a77-40a9-8086-9e85698b53a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.724034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-de06372d-8a77-40a9-8086-9e85698b53a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.737183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06372d-8a77-40a9-8086-9e85698b53a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.738677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06372d-8a77-40a9-8086-9e85698b53a3 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.739366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fe7cf66f-d55e-4779-9104-5f81f01b0910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffe0758ed24d4832a544df58efc3fa35'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.740313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fe7cf66f-d55e-4779-9104-5f81f01b0910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffe0758ed24d4832a544df58efc3fa35'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.741004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fe7cf66f-d55e-4779-9104-5f81f01b0910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffe0758ed24d4832a544df58efc3fa35'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.750297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fe7cf66f-d55e-4779-9104-5f81f01b0910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffe0758ed24d4832a544df58efc3fa35'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.750517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 944/1870] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:37.752161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e765439-6934-4dd7-9927-e29d0b6f6c70 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.752456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e765439-6934-4dd7-9927-e29d0b6f6c70 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.754232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e765439-6934-4dd7-9927-e29d0b6f6c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.759266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fe7cf66f-d55e-4779-9104-5f81f01b0910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffe0758ed24d4832a544df58efc3fa35'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:37.776421 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-9e765439-6934-4dd7-9927-e29d0b6f6c70 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:37.777632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9e765439-6934-4dd7-9927-e29d0b6f6c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.786649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 945/1871] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:37.787617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e765439-6934-4dd7-9927-e29d0b6f6c70 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.788804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9e765439-6934-4dd7-9927-e29d0b6f6c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.801182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 927/1872] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:37 2022] DELETE /identity/v3/users/ffe0758ed24d4832a544df58efc3fa35 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:37.802800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9f4df-ffa6-43c3-9b80-13bf91fe68e7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.803094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9f4df-ffa6-43c3-9b80-13bf91fe68e7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.803373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9f4df-ffa6-43c3-9b80-13bf91fe68e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.809153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e765439-6934-4dd7-9927-e29d0b6f6c70 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.835931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-eda9f4df-ffa6-43c3-9b80-13bf91fe68e7 None None] MFA Rules not processed for user `1ec901d609804e9fa8be4bbab3b03d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:37.837569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eda9f4df-ffa6-43c3-9b80-13bf91fe68e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.870253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d320f0b-206f-4c88-a829-3077cee986cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.870972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d320f0b-206f-4c88-a829-3077cee986cf service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.871290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d320f0b-206f-4c88-a829-3077cee986cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.871645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d320f0b-206f-4c88-a829-3077cee986cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.873691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7d320f0b-206f-4c88-a829-3077cee986cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.889743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:37.894760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:37.949651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d320f0b-206f-4c88-a829-3077cee986cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.951183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d320f0b-206f-4c88-a829-3077cee986cf service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.963117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 946/1873] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:37.964074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d320f0b-206f-4c88-a829-3077cee986cf service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.967402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 928/1874] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:37 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:37.976027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d320f0b-206f-4c88-a829-3077cee986cf service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.981215 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a571a-54f1-4cee-a441-d1f9a5d2b0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.981833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a571a-54f1-4cee-a441-d1f9a5d2b0cf admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.982263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a571a-54f1-4cee-a441-d1f9a5d2b0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.982548 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a571a-54f1-4cee-a441-d1f9a5d2b0cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.983213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a571a-54f1-4cee-a441-d1f9a5d2b0cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:37.983521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a571a-54f1-4cee-a441-d1f9a5d2b0cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.983848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9f4df-ffa6-43c3-9b80-13bf91fe68e7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.985242 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a571a-54f1-4cee-a441-d1f9a5d2b0cf admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.989170 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eda9f4df-ffa6-43c3-9b80-13bf91fe68e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.001464 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 947/1875] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:38.003472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a571a-54f1-4cee-a441-d1f9a5d2b0cf admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.021958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a571a-54f1-4cee-a441-d1f9a5d2b0cf admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.027553 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895f22e-c05a-481f-8dfa-e88e57b1f54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.028257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895f22e-c05a-481f-8dfa-e88e57b1f54c admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.028539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895f22e-c05a-481f-8dfa-e88e57b1f54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.028874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895f22e-c05a-481f-8dfa-e88e57b1f54c admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.029518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895f22e-c05a-481f-8dfa-e88e57b1f54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.032364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895f22e-c05a-481f-8dfa-e88e57b1f54c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.089694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 948/1876] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:38 2022] POST /identity/v3/projects => generated 387 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:38.094767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895f22e-c05a-481f-8dfa-e88e57b1f54c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.097564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9f4df-ffa6-43c3-9b80-13bf91fe68e7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.198451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895f22e-c05a-481f-8dfa-e88e57b1f54c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.201724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73fdf5-ca36-45de-ad19-9413d69bf84d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.203169 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73fdf5-ca36-45de-ad19-9413d69bf84d service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.203616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73fdf5-ca36-45de-ad19-9413d69bf84d service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.204527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73fdf5-ca36-45de-ad19-9413d69bf84d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.208949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615b2c7-cbde-4398-bdd5-a9b45c6fbceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.209735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615b2c7-cbde-4398-bdd5-a9b45c6fbceb admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.210117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615b2c7-cbde-4398-bdd5-a9b45c6fbceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.210520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615b2c7-cbde-4398-bdd5-a9b45c6fbceb admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.211509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615b2c7-cbde-4398-bdd5-a9b45c6fbceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.213866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615b2c7-cbde-4398-bdd5-a9b45c6fbceb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.252836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 949/1877] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:38 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:38.254451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615b2c7-cbde-4398-bdd5-a9b45c6fbceb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.271134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615b2c7-cbde-4398-bdd5-a9b45c6fbceb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.279975 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb7454-635a-4a94-a2c5-f1958036b597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.280799 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb7454-635a-4a94-a2c5-f1958036b597 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.281183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb7454-635a-4a94-a2c5-f1958036b597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.281855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb7454-635a-4a94-a2c5-f1958036b597 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.282695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb7454-635a-4a94-a2c5-f1958036b597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.284511 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb7454-635a-4a94-a2c5-f1958036b597 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.303837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 950/1878] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:38.309327 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb7454-635a-4a94-a2c5-f1958036b597 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.323462 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73fdf5-ca36-45de-ad19-9413d69bf84d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.324996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73fdf5-ca36-45de-ad19-9413d69bf84d service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.325807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb7454-635a-4a94-a2c5-f1958036b597 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.333986 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496b76d-ccea-4b11-bed7-5e98bb86bf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.334773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496b76d-ccea-4b11-bed7-5e98bb86bf6a admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.335866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496b76d-ccea-4b11-bed7-5e98bb86bf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.337127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496b76d-ccea-4b11-bed7-5e98bb86bf6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.337987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496b76d-ccea-4b11-bed7-5e98bb86bf6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.342833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496b76d-ccea-4b11-bed7-5e98bb86bf6a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.351694 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 929/1879] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:37 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:38.381254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 951/1880] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:38.383552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73fdf5-ca36-45de-ad19-9413d69bf84d service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.398594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73fdf5-ca36-45de-ad19-9413d69bf84d service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.417800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eda5ad-d7a1-41ab-9bbc-62fbce864b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.417800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eda5ad-d7a1-41ab-9bbc-62fbce864b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.417800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eda5ad-d7a1-41ab-9bbc-62fbce864b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.417800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eda5ad-d7a1-41ab-9bbc-62fbce864b32 admin admin] PATH_INFO: `/v3/projects/b5ada53236bf487f96dd65347de3ebc2/users/5617a0faa6b14b96948b9df385607ae4/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.428169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496b76d-ccea-4b11-bed7-5e98bb86bf6a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.430693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2496b76d-ccea-4b11-bed7-5e98bb86bf6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.449364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496b76d-ccea-4b11-bed7-5e98bb86bf6a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.453754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eda5ad-d7a1-41ab-9bbc-62fbce864b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ada53236bf487f96dd65347de3ebc2, user_id=5617a0faa6b14b96948b9df385607ae4, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.456469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eda5ad-d7a1-41ab-9bbc-62fbce864b32 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.475704 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83be7a-564a-4482-bda1-29d24aceed3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.476895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83be7a-564a-4482-bda1-29d24aceed3f service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.477371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83be7a-564a-4482-bda1-29d24aceed3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.477939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83be7a-564a-4482-bda1-29d24aceed3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.482739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ec83be7a-564a-4482-bda1-29d24aceed3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.490724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 930/1881] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:38 2022] PUT /identity/v3/projects/b5ada53236bf487f96dd65347de3ebc2/users/5617a0faa6b14b96948b9df385607ae4/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:38.503442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5701c2-f57c-4f81-9710-bbe8e956a554 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.503726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5701c2-f57c-4f81-9710-bbe8e956a554 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.503997 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5701c2-f57c-4f81-9710-bbe8e956a554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.550575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:38.560080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:38.571677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0c5701c2-f57c-4f81-9710-bbe8e956a554 None None] MFA Rules not processed for user `5617a0faa6b14b96948b9df385607ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:38.580399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0c5701c2-f57c-4f81-9710-bbe8e956a554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.595070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83be7a-564a-4482-bda1-29d24aceed3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.597254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83be7a-564a-4482-bda1-29d24aceed3f service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.613808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 952/1882] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:38 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:38.625987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83be7a-564a-4482-bda1-29d24aceed3f service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.629514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ec83be7a-564a-4482-bda1-29d24aceed3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.672290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83be7a-564a-4482-bda1-29d24aceed3f service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.782713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.783291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.783574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.783853 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] PATH_INFO: `/v3/projects/d7e9a17f65ca488e8939acad5497831a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.786862 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7e9a17f65ca488e8939acad5497831a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.788854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.798121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 931/1883] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:38.799008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5701c2-f57c-4f81-9710-bbe8e956a554 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.802278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0c5701c2-f57c-4f81-9710-bbe8e956a554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.832548 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5701c2-f57c-4f81-9710-bbe8e956a554 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.848417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.848733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.849150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.849530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] PATH_INFO: `/v3/projects/024d85dd05ad4caf8cf52efe7940ba0f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.850884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=024d85dd05ad4caf8cf52efe7940ba0f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.853695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.896778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7e9a17f65ca488e8939acad5497831a', 'request_id': 'req-78fb7913-61a9-4b64-8101-d159c4caf617'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.910772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7e9a17f65ca488e8939acad5497831a', 'request_id': 'req-78fb7913-61a9-4b64-8101-d159c4caf617'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.936077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7e9a17f65ca488e8939acad5497831a', 'request_id': 'req-78fb7913-61a9-4b64-8101-d159c4caf617'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.960164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '024d85dd05ad4caf8cf52efe7940ba0f', 'request_id': 'req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.963386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] The token cache is being invalidate because project d7e9a17f65ca488e8939acad5497831a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:38.963705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.967180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.968772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 953/1884] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:38 2022] DELETE /identity/v3/projects/d7e9a17f65ca488e8939acad5497831a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:38.969183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '024d85dd05ad4caf8cf52efe7940ba0f', 'request_id': 'req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.971034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.973224 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.979425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '024d85dd05ad4caf8cf52efe7940ba0f', 'request_id': 'req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.998340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] The token cache is being invalidate because project 024d85dd05ad4caf8cf52efe7940ba0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:39.000261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.001841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.007010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 932/1885] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:38 2022] DELETE /identity/v3/projects/024d85dd05ad4caf8cf52efe7940ba0f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:39.007010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.008983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.096901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb7913-61a9-4b64-8101-d159c4caf617 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.113461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.114315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.114541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ce8f3-cb56-46f8-bda8-f8b0c39e96f1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.114694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.114970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] PATH_INFO: `/v3/projects/e8d67c4463784059b12bdc3f332adca5` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.116663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8d67c4463784059b12bdc3f332adca5)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.119766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.122915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bae5b7-128f-43fe-bc33-2cf33292bcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.124913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bae5b7-128f-43fe-bc33-2cf33292bcca admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.125243 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bae5b7-128f-43fe-bc33-2cf33292bcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.125679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bae5b7-128f-43fe-bc33-2cf33292bcca admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.126591 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bae5b7-128f-43fe-bc33-2cf33292bcca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.129157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bae5b7-128f-43fe-bc33-2cf33292bcca admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.133196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:39.189743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 933/1886] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:39 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:39.200143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb87453-585f-45c0-a752-4b1b1a2e1f96 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.200143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb87453-585f-45c0-a752-4b1b1a2e1f96 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.200143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb87453-585f-45c0-a752-4b1b1a2e1f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.206112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d67c4463784059b12bdc3f332adca5', 'request_id': 'req-1a9d6653-8e11-44a9-b82e-103c0abdb023'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.215196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d67c4463784059b12bdc3f332adca5', 'request_id': 'req-1a9d6653-8e11-44a9-b82e-103c0abdb023'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.216738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ebb87453-585f-45c0-a752-4b1b1a2e1f96 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:39.217959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ebb87453-585f-45c0-a752-4b1b1a2e1f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.223030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:39.225037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d67c4463784059b12bdc3f332adca5', 'request_id': 'req-1a9d6653-8e11-44a9-b82e-103c0abdb023'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.246091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] The token cache is being invalidate because project e8d67c4463784059b12bdc3f332adca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:39.246386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.247189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a9d6653-8e11-44a9-b82e-103c0abdb023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.248868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 954/1887] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:38 2022] DELETE /identity/v3/projects/e8d67c4463784059b12bdc3f332adca5 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:39.250624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae99808-3137-4789-9ff6-38a6477e9661 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.250908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae99808-3137-4789-9ff6-38a6477e9661 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.251178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae99808-3137-4789-9ff6-38a6477e9661 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.252860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 955/1888] 10.222.0.86 () {54 vars in 921 bytes} [Wed Aug 24 10:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:03:39.253146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae99808-3137-4789-9ff6-38a6477e9661 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.255580 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fae99808-3137-4789-9ff6-38a6477e9661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.303350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 934/1889] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:39.304384 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb87453-585f-45c0-a752-4b1b1a2e1f96 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.312565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ebb87453-585f-45c0-a752-4b1b1a2e1f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.326554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb87453-585f-45c0-a752-4b1b1a2e1f96 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.332657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a30e74-0075-4a7b-8bd8-0488a3d8ed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.333301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a30e74-0075-4a7b-8bd8-0488a3d8ed7b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.333581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a30e74-0075-4a7b-8bd8-0488a3d8ed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.333852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a30e74-0075-4a7b-8bd8-0488a3d8ed7b admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.334756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a30e74-0075-4a7b-8bd8-0488a3d8ed7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.336440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a30e74-0075-4a7b-8bd8-0488a3d8ed7b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.343297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae99808-3137-4789-9ff6-38a6477e9661 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.348926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f0afc-d8c2-4454-8a03-87c7db6d704a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.349662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f0afc-d8c2-4454-8a03-87c7db6d704a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.349972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f0afc-d8c2-4454-8a03-87c7db6d704a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.350279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f0afc-d8c2-4454-8a03-87c7db6d704a admin admin] PATH_INFO: `/v3/users/9430d6b1327b4fefaf230a89edd8f413` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.351522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f0afc-d8c2-4454-8a03-87c7db6d704a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9430d6b1327b4fefaf230a89edd8f413)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.353654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f0afc-d8c2-4454-8a03-87c7db6d704a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.359783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 935/1890] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:39 2022] POST /identity/v3/users => generated 293 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:39.361255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a30e74-0075-4a7b-8bd8-0488a3d8ed7b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.363894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-71a30e74-0075-4a7b-8bd8-0488a3d8ed7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.376659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a30e74-0075-4a7b-8bd8-0488a3d8ed7b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.381632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbed1a-5d36-481c-8abc-bda10287a121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.382326 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbed1a-5d36-481c-8abc-bda10287a121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.382524 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbed1a-5d36-481c-8abc-bda10287a121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.382795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbed1a-5d36-481c-8abc-bda10287a121 admin admin] PATH_INFO: `/v3/users/8603306cc97b45d3aa2788cf0383bbf2` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.384235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbed1a-5d36-481c-8abc-bda10287a121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8603306cc97b45d3aa2788cf0383bbf2)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.386255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbed1a-5d36-481c-8abc-bda10287a121 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.407541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-949f0afc-d8c2-4454-8a03-87c7db6d704a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9430d6b1327b4fefaf230a89edd8f413'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.408238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-949f0afc-d8c2-4454-8a03-87c7db6d704a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9430d6b1327b4fefaf230a89edd8f413'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.408876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-949f0afc-d8c2-4454-8a03-87c7db6d704a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9430d6b1327b4fefaf230a89edd8f413'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.416784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-949f0afc-d8c2-4454-8a03-87c7db6d704a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9430d6b1327b4fefaf230a89edd8f413'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.425152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-949f0afc-d8c2-4454-8a03-87c7db6d704a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9430d6b1327b4fefaf230a89edd8f413'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.434511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eddbed1a-5d36-481c-8abc-bda10287a121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8603306cc97b45d3aa2788cf0383bbf2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.435172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eddbed1a-5d36-481c-8abc-bda10287a121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8603306cc97b45d3aa2788cf0383bbf2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.436140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eddbed1a-5d36-481c-8abc-bda10287a121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8603306cc97b45d3aa2788cf0383bbf2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.445236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eddbed1a-5d36-481c-8abc-bda10287a121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8603306cc97b45d3aa2788cf0383bbf2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.451339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 956/1891] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:39 2022] DELETE /identity/v3/users/9430d6b1327b4fefaf230a89edd8f413 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:39.452871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7029751-abc4-4f4d-98bc-ecfb22c7374a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.453257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7029751-abc4-4f4d-98bc-ecfb22c7374a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.453606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7029751-abc4-4f4d-98bc-ecfb22c7374a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.455346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eddbed1a-5d36-481c-8abc-bda10287a121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8603306cc97b45d3aa2788cf0383bbf2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.474832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a7029751-abc4-4f4d-98bc-ecfb22c7374a None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:39.476251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a7029751-abc4-4f4d-98bc-ecfb22c7374a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.479729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 936/1892] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:39 2022] DELETE /identity/v3/users/8603306cc97b45d3aa2788cf0383bbf2 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:39.480823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c6848-f7a1-4b50-ba66-698fc2d2c863 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.481257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c6848-f7a1-4b50-ba66-698fc2d2c863 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.481545 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c6848-f7a1-4b50-ba66-698fc2d2c863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.494693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:39.507130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0d7c6848-f7a1-4b50-ba66-698fc2d2c863 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:39.508725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0d7c6848-f7a1-4b50-ba66-698fc2d2c863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.572723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 957/1893] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Aug 24 10:03:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:39.574131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7029751-abc4-4f4d-98bc-ecfb22c7374a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.575371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a7029751-abc4-4f4d-98bc-ecfb22c7374a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.598301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 937/1894] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:39.601893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c6848-f7a1-4b50-ba66-698fc2d2c863 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.601893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0d7c6848-f7a1-4b50-ba66-698fc2d2c863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.615848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:39.621692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c6848-f7a1-4b50-ba66-698fc2d2c863 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.631580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c02ba-9e1b-4b5c-a3b2-1355e4b57a8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.633111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c02ba-9e1b-4b5c-a3b2-1355e4b57a8f service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.633968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c02ba-9e1b-4b5c-a3b2-1355e4b57a8f service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.633968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c02ba-9e1b-4b5c-a3b2-1355e4b57a8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.648847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c02ba-9e1b-4b5c-a3b2-1355e4b57a8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.651515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c02ba-9e1b-4b5c-a3b2-1355e4b57a8f service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.675026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 938/1895] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:39.679253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c02ba-9e1b-4b5c-a3b2-1355e4b57a8f service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.704030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c02ba-9e1b-4b5c-a3b2-1355e4b57a8f service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.724056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7029751-abc4-4f4d-98bc-ecfb22c7374a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.730266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc030bf-a5ec-4a58-893c-fbc79f673f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.731168 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc030bf-a5ec-4a58-893c-fbc79f673f6f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.731492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc030bf-a5ec-4a58-893c-fbc79f673f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.731838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc030bf-a5ec-4a58-893c-fbc79f673f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.732603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc030bf-a5ec-4a58-893c-fbc79f673f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.733190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fff152-7b57-4cb1-aadb-678d5f567065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.734478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fff152-7b57-4cb1-aadb-678d5f567065 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.734862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fff152-7b57-4cb1-aadb-678d5f567065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.735104 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc030bf-a5ec-4a58-893c-fbc79f673f6f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.735580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fff152-7b57-4cb1-aadb-678d5f567065 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.736380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fff152-7b57-4cb1-aadb-678d5f567065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:39.737474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fff152-7b57-4cb1-aadb-678d5f567065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.740500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fff152-7b57-4cb1-aadb-678d5f567065 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.769324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 939/1896] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:39.769515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 958/1897] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:39.771127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fff152-7b57-4cb1-aadb-678d5f567065 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.771744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc030bf-a5ec-4a58-893c-fbc79f673f6f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.794808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fff152-7b57-4cb1-aadb-678d5f567065 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.810988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19ac5a-9ac2-46cd-8e26-bb9336894597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.811706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19ac5a-9ac2-46cd-8e26-bb9336894597 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.812247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19ac5a-9ac2-46cd-8e26-bb9336894597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.812664 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19ac5a-9ac2-46cd-8e26-bb9336894597 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.813051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc030bf-a5ec-4a58-893c-fbc79f673f6f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.813499 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19ac5a-9ac2-46cd-8e26-bb9336894597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.815601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19ac5a-9ac2-46cd-8e26-bb9336894597 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.845194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aa1ed-9b0a-4b7d-b99c-ceeccd462dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.846791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aa1ed-9b0a-4b7d-b99c-ceeccd462dba admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.848074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aa1ed-9b0a-4b7d-b99c-ceeccd462dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.848507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aa1ed-9b0a-4b7d-b99c-ceeccd462dba admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.851290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685aa1ed-9b0a-4b7d-b99c-ceeccd462dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.857555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685aa1ed-9b0a-4b7d-b99c-ceeccd462dba admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.876343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 940/1898] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:39 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:39.880325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19ac5a-9ac2-46cd-8e26-bb9336894597 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.896244 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 959/1899] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:39.900678 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aa1ed-9b0a-4b7d-b99c-ceeccd462dba admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.929430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aa1ed-9b0a-4b7d-b99c-ceeccd462dba admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.024834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a93c99-c4e1-44bf-bc5f-32c67e1a3e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.026410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93c99-c4e1-44bf-bc5f-32c67e1a3e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.026888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93c99-c4e1-44bf-bc5f-32c67e1a3e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.027339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93c99-c4e1-44bf-bc5f-32c67e1a3e88 admin admin] PATH_INFO: `/v3/projects/067a44dec03945cfab82b08094dfbf46/users/b1df7fe3dcf244ada3a2b25ac86853b3/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.049584 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a93c99-c4e1-44bf-bc5f-32c67e1a3e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=067a44dec03945cfab82b08094dfbf46, user_id=b1df7fe3dcf244ada3a2b25ac86853b3, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.052164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19ac5a-9ac2-46cd-8e26-bb9336894597 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.055791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a93c99-c4e1-44bf-bc5f-32c67e1a3e88 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.066001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701a5bd-7e5b-4812-b45e-4c216d901615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.066792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701a5bd-7e5b-4812-b45e-4c216d901615 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.067180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701a5bd-7e5b-4812-b45e-4c216d901615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.068226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701a5bd-7e5b-4812-b45e-4c216d901615 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.068987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701a5bd-7e5b-4812-b45e-4c216d901615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.071070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701a5bd-7e5b-4812-b45e-4c216d901615 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.078532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 960/1900] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:39 2022] PUT /identity/v3/projects/067a44dec03945cfab82b08094dfbf46/users/b1df7fe3dcf244ada3a2b25ac86853b3/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:40.089784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d2385-f7c8-463b-b8a7-a79a66781659 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.090417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d2385-f7c8-463b-b8a7-a79a66781659 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.090847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d2385-f7c8-463b-b8a7-a79a66781659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.112101 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 941/1901] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:39 2022] POST /identity/v3/users => generated 283 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:40.113791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701a5bd-7e5b-4812-b45e-4c216d901615 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.149061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701a5bd-7e5b-4812-b45e-4c216d901615 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.181603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-af9d2385-f7c8-463b-b8a7-a79a66781659 None None] MFA Rules not processed for user `b1df7fe3dcf244ada3a2b25ac86853b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:40.183995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-af9d2385-f7c8-463b-b8a7-a79a66781659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.231724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5ed33-3ad3-40f1-8e17-56fef814c015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.232604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ed33-3ad3-40f1-8e17-56fef814c015 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.232991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ed33-3ad3-40f1-8e17-56fef814c015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.233372 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ed33-3ad3-40f1-8e17-56fef814c015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.234618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5ed33-3ad3-40f1-8e17-56fef814c015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.236753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5ed33-3ad3-40f1-8e17-56fef814c015 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.238943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:40.250240 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 942/1902] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:40.251886 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5ed33-3ad3-40f1-8e17-56fef814c015 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.254853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-04c5ed33-3ad3-40f1-8e17-56fef814c015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.269836 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5ed33-3ad3-40f1-8e17-56fef814c015 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.275967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.276758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.277993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.278382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] PATH_INFO: `/v3/projects/fe5205183bbd457a8b84475c5c9582d3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.279693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5205183bbd457a8b84475c5c9582d3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.282944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.363485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:40.389386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5205183bbd457a8b84475c5c9582d3', 'request_id': 'req-9a0f0120-46f4-4a50-8896-1269a18813ff'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.394366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 961/1903] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:40.394366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d2385-f7c8-463b-b8a7-a79a66781659 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.399547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5205183bbd457a8b84475c5c9582d3', 'request_id': 'req-9a0f0120-46f4-4a50-8896-1269a18813ff'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.408319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5205183bbd457a8b84475c5c9582d3', 'request_id': 'req-9a0f0120-46f4-4a50-8896-1269a18813ff'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.420338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d2385-f7c8-463b-b8a7-a79a66781659 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.434681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] The token cache is being invalidate because project fe5205183bbd457a8b84475c5c9582d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:40.435354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.436308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.438386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 943/1904] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:40 2022] DELETE /identity/v3/projects/fe5205183bbd457a8b84475c5c9582d3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:40.439919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.441383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.517870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4ed43-7242-46c9-bbde-afd5e6714387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.519512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4ed43-7242-46c9-bbde-afd5e6714387 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.519512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4ed43-7242-46c9-bbde-afd5e6714387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.519512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4ed43-7242-46c9-bbde-afd5e6714387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.520309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4ed43-7242-46c9-bbde-afd5e6714387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.522734 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4ed43-7242-46c9-bbde-afd5e6714387 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.538633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 962/1905] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:40.539723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4ed43-7242-46c9-bbde-afd5e6714387 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.543277 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fbf4ed43-7242-46c9-bbde-afd5e6714387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.560009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4ed43-7242-46c9-bbde-afd5e6714387 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.569038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f0120-46f4-4a50-8896-1269a18813ff admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.569282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cec45c-6a2a-4fc4-8aca-3404dd31c8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.570002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cec45c-6a2a-4fc4-8aca-3404dd31c8b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.570296 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cec45c-6a2a-4fc4-8aca-3404dd31c8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.570559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cec45c-6a2a-4fc4-8aca-3404dd31c8b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.571631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cec45c-6a2a-4fc4-8aca-3404dd31c8b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.574434 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cec45c-6a2a-4fc4-8aca-3404dd31c8b9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.575066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.575966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.576789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.576789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] PATH_INFO: `/v3/projects/be338b2137bb448facb68975b9b3262a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.578741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be338b2137bb448facb68975b9b3262a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.581094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.607746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 963/1906] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:40.609394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e39745-6d86-41ef-a1a0-3fc9893ba269 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.609713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e39745-6d86-41ef-a1a0-3fc9893ba269 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.610058 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e39745-6d86-41ef-a1a0-3fc9893ba269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.639006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f1e39745-6d86-41ef-a1a0-3fc9893ba269 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:40.640897 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f1e39745-6d86-41ef-a1a0-3fc9893ba269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.648639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be338b2137bb448facb68975b9b3262a', 'request_id': 'req-9aead3db-18de-4903-926b-860b2c7057e0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.656376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be338b2137bb448facb68975b9b3262a', 'request_id': 'req-9aead3db-18de-4903-926b-860b2c7057e0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.663459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be338b2137bb448facb68975b9b3262a', 'request_id': 'req-9aead3db-18de-4903-926b-860b2c7057e0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.681877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] The token cache is being invalidate because project be338b2137bb448facb68975b9b3262a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:40.683036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.684309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.688082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 944/1907] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:40 2022] DELETE /identity/v3/projects/be338b2137bb448facb68975b9b3262a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:40.689149 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.691017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.737982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 964/1908] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:40.738969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e39745-6d86-41ef-a1a0-3fc9893ba269 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.740492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f1e39745-6d86-41ef-a1a0-3fc9893ba269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.756759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e39745-6d86-41ef-a1a0-3fc9893ba269 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.764670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2c358-8da8-45d7-8fdc-a3fab3fec894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.765802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2c358-8da8-45d7-8fdc-a3fab3fec894 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.766076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2c358-8da8-45d7-8fdc-a3fab3fec894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.768156 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2c358-8da8-45d7-8fdc-a3fab3fec894 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.768156 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2c358-8da8-45d7-8fdc-a3fab3fec894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.770387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2c358-8da8-45d7-8fdc-a3fab3fec894 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.796909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 965/1909] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/users => generated 299 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:40.798842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac22805-3e7e-4591-a39a-0cdaf7901016 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.799229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac22805-3e7e-4591-a39a-0cdaf7901016 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.799777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac22805-3e7e-4591-a39a-0cdaf7901016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.803227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:40.809327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aead3db-18de-4903-926b-860b2c7057e0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.813652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:40.823944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c9dcd-d529-4359-8da2-036b8270b83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.826049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c9dcd-d529-4359-8da2-036b8270b83e admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.826327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c9dcd-d529-4359-8da2-036b8270b83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.826605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c9dcd-d529-4359-8da2-036b8270b83e admin admin] PATH_INFO: `/v3/projects/fa4926f62db74724bbb8ba274402b10d/users/1bffd6da32a94611ace43d6eada4cae8/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.826854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cac22805-3e7e-4591-a39a-0cdaf7901016 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:40.828712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cac22805-3e7e-4591-a39a-0cdaf7901016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.843253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 966/1910] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:40.844381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127934b-d944-4119-ab6c-8d136691e189 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.844666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127934b-d944-4119-ab6c-8d136691e189 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.845086 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127934b-d944-4119-ab6c-8d136691e189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.849941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c9dcd-d529-4359-8da2-036b8270b83e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4926f62db74724bbb8ba274402b10d, user_id=1bffd6da32a94611ace43d6eada4cae8, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.853057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c9dcd-d529-4359-8da2-036b8270b83e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.872501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e127934b-d944-4119-ab6c-8d136691e189 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:40.873841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e127934b-d944-4119-ab6c-8d136691e189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.874868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 945/1911] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:40 2022] PUT /identity/v3/projects/fa4926f62db74724bbb8ba274402b10d/users/1bffd6da32a94611ace43d6eada4cae8/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:40.875855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c9dcd-d529-4359-8da2-036b8270b83e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.971062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 967/1912] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:40.972387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f1d65-ce63-411d-be57-c543cd318210 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.973240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f1d65-ce63-411d-be57-c543cd318210 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.973240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f1d65-ce63-411d-be57-c543cd318210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.976130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c9dcd-d529-4359-8da2-036b8270b83e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.988110 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf5995-70aa-44b7-96c4-0bf412f71d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.988849 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5995-70aa-44b7-96c4-0bf412f71d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.989232 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5995-70aa-44b7-96c4-0bf412f71d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.989684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5995-70aa-44b7-96c4-0bf412f71d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.990581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf5995-70aa-44b7-96c4-0bf412f71d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.992465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf5995-70aa-44b7-96c4-0bf412f71d2c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.999294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-387f1d65-ce63-411d-be57-c543cd318210 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:41.001309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-387f1d65-ce63-411d-be57-c543cd318210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:41.005330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 946/1913] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:41.014196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 968/1914] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:41.015478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f1d65-ce63-411d-be57-c543cd318210 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.018533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32fc77-c169-4dba-81e8-467327f556d9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.018829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32fc77-c169-4dba-81e8-467327f556d9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.019107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32fc77-c169-4dba-81e8-467327f556d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.030309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f1d65-ce63-411d-be57-c543cd318210 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.037382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa160d5c-3828-462f-942f-116afc85a334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.038643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa160d5c-3828-462f-942f-116afc85a334 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.038927 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa160d5c-3828-462f-942f-116afc85a334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.039276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa160d5c-3828-462f-942f-116afc85a334 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.040185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa160d5c-3828-462f-942f-116afc85a334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:41.040481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa160d5c-3828-462f-942f-116afc85a334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.043869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa160d5c-3828-462f-942f-116afc85a334 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.052438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5d32fc77-c169-4dba-81e8-467327f556d9 None None] MFA Rules not processed for user `1bffd6da32a94611ace43d6eada4cae8`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:41.053753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5d32fc77-c169-4dba-81e8-467327f556d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:41.060180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 969/1915] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:41.061141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa160d5c-3828-462f-942f-116afc85a334 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.072945 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa160d5c-3828-462f-942f-116afc85a334 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.077776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34130af7-c67b-4197-91d6-4a268f4a24cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.078433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34130af7-c67b-4197-91d6-4a268f4a24cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.078718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34130af7-c67b-4197-91d6-4a268f4a24cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.078996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34130af7-c67b-4197-91d6-4a268f4a24cc admin admin] PATH_INFO: `/v3/projects/58a3f2c2e4164917a6413f44dfe7c006/users/d92424f4248e4d06b8755bb4d074e69c/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.094301 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34130af7-c67b-4197-91d6-4a268f4a24cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a3f2c2e4164917a6413f44dfe7c006, user_id=d92424f4248e4d06b8755bb4d074e69c, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.097097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34130af7-c67b-4197-91d6-4a268f4a24cc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.112595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 970/1916] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:41 2022] PUT /identity/v3/projects/58a3f2c2e4164917a6413f44dfe7c006/users/d92424f4248e4d06b8755bb4d074e69c/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:41.113248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34130af7-c67b-4197-91d6-4a268f4a24cc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.135267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 947/1917] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:41.136183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32fc77-c169-4dba-81e8-467327f556d9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.148599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32fc77-c169-4dba-81e8-467327f556d9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.213274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34130af7-c67b-4197-91d6-4a268f4a24cc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.217328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748837f7-89f8-4f7a-8e0a-908e44c0b343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.217765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748837f7-89f8-4f7a-8e0a-908e44c0b343 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.218618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748837f7-89f8-4f7a-8e0a-908e44c0b343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.218951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748837f7-89f8-4f7a-8e0a-908e44c0b343 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.219227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbada8cf-3edf-4111-9aa2-f795ce571a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.219645 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748837f7-89f8-4f7a-8e0a-908e44c0b343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.220025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbada8cf-3edf-4111-9aa2-f795ce571a44 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.221390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbada8cf-3edf-4111-9aa2-f795ce571a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.221767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbada8cf-3edf-4111-9aa2-f795ce571a44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.222518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748837f7-89f8-4f7a-8e0a-908e44c0b343 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.222728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbada8cf-3edf-4111-9aa2-f795ce571a44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:41.223029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbada8cf-3edf-4111-9aa2-f795ce571a44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.226408 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbada8cf-3edf-4111-9aa2-f795ce571a44 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.239326 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:41.248077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 971/1918] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:41.249207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbada8cf-3edf-4111-9aa2-f795ce571a44 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.263991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbada8cf-3edf-4111-9aa2-f795ce571a44 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.270050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7036e9-b8bb-4959-89df-eb290c16aeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.270841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7036e9-b8bb-4959-89df-eb290c16aeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.271261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7036e9-b8bb-4959-89df-eb290c16aeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.271672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7036e9-b8bb-4959-89df-eb290c16aeb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.272899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7036e9-b8bb-4959-89df-eb290c16aeb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.277618 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7036e9-b8bb-4959-89df-eb290c16aeb6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.285600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 948/1919] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:41 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:41.285600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748837f7-89f8-4f7a-8e0a-908e44c0b343 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.297483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 972/1920] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:41.298394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7036e9-b8bb-4959-89df-eb290c16aeb6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.324525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7036e9-b8bb-4959-89df-eb290c16aeb6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.360730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748837f7-89f8-4f7a-8e0a-908e44c0b343 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.361209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:41.366543 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38577b9-b876-4f54-9759-25e595ab7e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.367162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38577b9-b876-4f54-9759-25e595ab7e86 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.367445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38577b9-b876-4f54-9759-25e595ab7e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.367729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38577b9-b876-4f54-9759-25e595ab7e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.368422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38577b9-b876-4f54-9759-25e595ab7e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.370610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38577b9-b876-4f54-9759-25e595ab7e86 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.414419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 949/1921] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:41 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:41.414419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38577b9-b876-4f54-9759-25e595ab7e86 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.417349 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7574d9-9962-4f70-8398-28d181e4d950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.417957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7574d9-9962-4f70-8398-28d181e4d950 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.418281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7574d9-9962-4f70-8398-28d181e4d950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.418538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7574d9-9962-4f70-8398-28d181e4d950 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.419178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7574d9-9962-4f70-8398-28d181e4d950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.421702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7574d9-9962-4f70-8398-28d181e4d950 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.466005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 973/1922] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:41 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:41.466005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7574d9-9962-4f70-8398-28d181e4d950 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.519670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38577b9-b876-4f54-9759-25e595ab7e86 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.551641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7574d9-9962-4f70-8398-28d181e4d950 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.557754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04257835-47b4-447b-97b0-79b10646b385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.558418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04257835-47b4-447b-97b0-79b10646b385 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.558795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04257835-47b4-447b-97b0-79b10646b385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.559078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04257835-47b4-447b-97b0-79b10646b385 admin admin] PATH_INFO: `/v3/projects/58a3f2c2e4164917a6413f44dfe7c006/users/d92424f4248e4d06b8755bb4d074e69c/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.561130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04257835-47b4-447b-97b0-79b10646b385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a3f2c2e4164917a6413f44dfe7c006, user_id=d92424f4248e4d06b8755bb4d074e69c, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.563898 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04257835-47b4-447b-97b0-79b10646b385 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.575064 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 974/1923] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:41 2022] PUT /identity/v3/projects/58a3f2c2e4164917a6413f44dfe7c006/users/d92424f4248e4d06b8755bb4d074e69c/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:41.575959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04257835-47b4-447b-97b0-79b10646b385 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.600207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd1a2a-59d7-429e-9cb6-945b2305723c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.600857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd1a2a-59d7-429e-9cb6-945b2305723c admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.601135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd1a2a-59d7-429e-9cb6-945b2305723c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.601410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd1a2a-59d7-429e-9cb6-945b2305723c admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.602040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd1a2a-59d7-429e-9cb6-945b2305723c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.603995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd1a2a-59d7-429e-9cb6-945b2305723c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.627903 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 950/1924] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:41 2022] POST /identity/v3/users => generated 291 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:41.628619 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd1a2a-59d7-429e-9cb6-945b2305723c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.638223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd1a2a-59d7-429e-9cb6-945b2305723c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.647743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:41.649969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04257835-47b4-447b-97b0-79b10646b385 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.658015 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4396784-f954-4420-acf4-531200edbbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.658641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4396784-f954-4420-acf4-531200edbbfd admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.659051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4396784-f954-4420-acf4-531200edbbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.659336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4396784-f954-4420-acf4-531200edbbfd admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.660087 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4396784-f954-4420-acf4-531200edbbfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.662030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4396784-f954-4420-acf4-531200edbbfd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.687743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 975/1925] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:41 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:41.692090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86587af-eb58-417e-ba00-aa37c8a83733 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.692386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86587af-eb58-417e-ba00-aa37c8a83733 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.692665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86587af-eb58-417e-ba00-aa37c8a83733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.709700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0b213-7500-4190-9c96-e1eb814c1c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.710324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0b213-7500-4190-9c96-e1eb814c1c99 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.710615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0b213-7500-4190-9c96-e1eb814c1c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.710892 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0b213-7500-4190-9c96-e1eb814c1c99 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.711526 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0b213-7500-4190-9c96-e1eb814c1c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.713512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0b213-7500-4190-9c96-e1eb814c1c99 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.728692 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b86587af-eb58-417e-ba00-aa37c8a83733 None None] MFA Rules not processed for user `d92424f4248e4d06b8755bb4d074e69c`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:41.730177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b86587af-eb58-417e-ba00-aa37c8a83733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:41.738172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 951/1926] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:41 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:41.739254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0b213-7500-4190-9c96-e1eb814c1c99 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.750377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0b213-7500-4190-9c96-e1eb814c1c99 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.755728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbef8fe-60f2-44a5-b101-92dfb82a9458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.756365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbef8fe-60f2-44a5-b101-92dfb82a9458 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.756644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbef8fe-60f2-44a5-b101-92dfb82a9458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.756917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbef8fe-60f2-44a5-b101-92dfb82a9458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.757560 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbef8fe-60f2-44a5-b101-92dfb82a9458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.759327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbef8fe-60f2-44a5-b101-92dfb82a9458 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.778110 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 952/1927] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:41.781448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:41.787690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbef8fe-60f2-44a5-b101-92dfb82a9458 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.801317 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbef8fe-60f2-44a5-b101-92dfb82a9458 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.806238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc8fbd-8381-4ece-b568-d93c4184f05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.806847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc8fbd-8381-4ece-b568-d93c4184f05b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.807123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc8fbd-8381-4ece-b568-d93c4184f05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.807391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc8fbd-8381-4ece-b568-d93c4184f05b admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.808114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc8fbd-8381-4ece-b568-d93c4184f05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.809903 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc8fbd-8381-4ece-b568-d93c4184f05b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.821366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 953/1928] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:41.822629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc8fbd-8381-4ece-b568-d93c4184f05b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.827944 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 976/1929] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:41.832339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86587af-eb58-417e-ba00-aa37c8a83733 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.835023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc8fbd-8381-4ece-b568-d93c4184f05b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.840387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e853fc-7ba9-485d-b4e8-4a122cc853ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.841042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e853fc-7ba9-485d-b4e8-4a122cc853ae admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.841331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e853fc-7ba9-485d-b4e8-4a122cc853ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.841610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e853fc-7ba9-485d-b4e8-4a122cc853ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.842396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e853fc-7ba9-485d-b4e8-4a122cc853ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.844622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86587af-eb58-417e-ba00-aa37c8a83733 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.844835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e853fc-7ba9-485d-b4e8-4a122cc853ae admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.850137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868beaf-4945-4c3c-b35d-3a735889eede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.851084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868beaf-4945-4c3c-b35d-3a735889eede admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.851366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868beaf-4945-4c3c-b35d-3a735889eede admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.851637 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868beaf-4945-4c3c-b35d-3a735889eede admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.852344 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868beaf-4945-4c3c-b35d-3a735889eede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.853946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868beaf-4945-4c3c-b35d-3a735889eede admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.855222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 954/1930] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:41.858993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e853fc-7ba9-485d-b4e8-4a122cc853ae admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.868727 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 977/1931] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:41.868727 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868beaf-4945-4c3c-b35d-3a735889eede admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.869563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e853fc-7ba9-485d-b4e8-4a122cc853ae admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.870203 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d868beaf-4945-4c3c-b35d-3a735889eede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:41.874365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e7706-2915-4b32-bc2d-da72bf349397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.875043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e7706-2915-4b32-bc2d-da72bf349397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.875330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e7706-2915-4b32-bc2d-da72bf349397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.875604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e7706-2915-4b32-bc2d-da72bf349397 admin admin] PATH_INFO: `/v3/projects/75ffe18832094f0bae945ae881e0136e/users/d0fdf6eee08f4ae48deda78cd4b52b51/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.890867 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e7706-2915-4b32-bc2d-da72bf349397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ffe18832094f0bae945ae881e0136e, user_id=d0fdf6eee08f4ae48deda78cd4b52b51, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.893620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e7706-2915-4b32-bc2d-da72bf349397 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.908193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 955/1932] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:41 2022] PUT /identity/v3/projects/75ffe18832094f0bae945ae881e0136e/users/d0fdf6eee08f4ae48deda78cd4b52b51/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:41.909016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e7706-2915-4b32-bc2d-da72bf349397 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.929289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868beaf-4945-4c3c-b35d-3a735889eede admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.982882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e7706-2915-4b32-bc2d-da72bf349397 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.986965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802c845-a09f-4744-ad98-fa1cbfc7fab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.987584 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802c845-a09f-4744-ad98-fa1cbfc7fab9 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.987863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802c845-a09f-4744-ad98-fa1cbfc7fab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.988178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802c845-a09f-4744-ad98-fa1cbfc7fab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.988392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b78d8-b9b5-440a-bbc8-2bbec1a5c6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.988854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b78d8-b9b5-440a-bbc8-2bbec1a5c6bc admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.989336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b78d8-b9b5-440a-bbc8-2bbec1a5c6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.989579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b78d8-b9b5-440a-bbc8-2bbec1a5c6bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.990308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b78d8-b9b5-440a-bbc8-2bbec1a5c6bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.992171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b78d8-b9b5-440a-bbc8-2bbec1a5c6bc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:42.009001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 956/1933] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:42.009001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b78d8-b9b5-440a-bbc8-2bbec1a5c6bc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:42.017265 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b78d8-b9b5-440a-bbc8-2bbec1a5c6bc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:42.025186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edd485-8c16-4e6b-999f-912dca6e1d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:42.026057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edd485-8c16-4e6b-999f-912dca6e1d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.026229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edd485-8c16-4e6b-999f-912dca6e1d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.026511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edd485-8c16-4e6b-999f-912dca6e1d56 admin admin] PATH_INFO: `/v3/projects/4c63db63b5e847c29d7a0618d7a9676b/users/72732b3547fa461ea5a0219dd7ea8537/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.043037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edd485-8c16-4e6b-999f-912dca6e1d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c63db63b5e847c29d7a0618d7a9676b, user_id=72732b3547fa461ea5a0219dd7ea8537, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:42.045779 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edd485-8c16-4e6b-999f-912dca6e1d56 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:42.059822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802c845-a09f-4744-ad98-fa1cbfc7fab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:42.060652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 957/1934] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:42 2022] PUT /identity/v3/projects/4c63db63b5e847c29d7a0618d7a9676b/users/72732b3547fa461ea5a0219dd7ea8537/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:42.061247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edd485-8c16-4e6b-999f-912dca6e1d56 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:42.062926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802c845-a09f-4744-ad98-fa1cbfc7fab9 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:42.078418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 978/1935] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:42.079336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802c845-a09f-4744-ad98-fa1cbfc7fab9 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:42.090167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802c845-a09f-4744-ad98-fa1cbfc7fab9 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:42.090781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:42.123178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:42.151332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edd485-8c16-4e6b-999f-912dca6e1d56 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:42.156999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf771e-e58f-4d6d-91a4-0f88e6b2dc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:42.158012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf771e-e58f-4d6d-91a4-0f88e6b2dc36 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.158373 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf771e-e58f-4d6d-91a4-0f88e6b2dc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.158831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf771e-e58f-4d6d-91a4-0f88e6b2dc36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.159555 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf771e-e58f-4d6d-91a4-0f88e6b2dc36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:42.161253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf771e-e58f-4d6d-91a4-0f88e6b2dc36 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:42.172000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325bead-f926-4d63-b46a-dc83c20f5cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:42.175412 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 958/1936] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:42.175556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325bead-f926-4d63-b46a-dc83c20f5cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.175556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325bead-f926-4d63-b46a-dc83c20f5cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.175556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325bead-f926-4d63-b46a-dc83c20f5cbb admin admin] PATH_INFO: `/v3/projects/e2d9c20d44c643518d2e6bd565f8ccda/users/75ef0e5dbd584576821298a35f1f5b05/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.175787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9ff86-baea-435a-853f-5eb251e478e9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.175787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9ff86-baea-435a-853f-5eb251e478e9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.175787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9ff86-baea-435a-853f-5eb251e478e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.190965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325bead-f926-4d63-b46a-dc83c20f5cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d9c20d44c643518d2e6bd565f8ccda, user_id=75ef0e5dbd584576821298a35f1f5b05, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:42.194436 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325bead-f926-4d63-b46a-dc83c20f5cbb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:42.211879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-24b9ff86-baea-435a-853f-5eb251e478e9 None None] MFA Rules not processed for user `72732b3547fa461ea5a0219dd7ea8537`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:42.213268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-24b9ff86-baea-435a-853f-5eb251e478e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:42.219335 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 979/1937] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:42 2022] PUT /identity/v3/projects/e2d9c20d44c643518d2e6bd565f8ccda/users/75ef0e5dbd584576821298a35f1f5b05/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:42.220830 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325bead-f926-4d63-b46a-dc83c20f5cbb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:42.303553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 959/1938] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:42.303553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee7cd4-d734-4470-844d-85cc94194d19 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.303553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee7cd4-d734-4470-844d-85cc94194d19 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.303553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee7cd4-d734-4470-844d-85cc94194d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.312705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325bead-f926-4d63-b46a-dc83c20f5cbb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:42.321523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbc943-0c94-4eb3-9811-1605b0b4feb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:42.322717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbc943-0c94-4eb3-9811-1605b0b4feb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.323198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbc943-0c94-4eb3-9811-1605b0b4feb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.323670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbc943-0c94-4eb3-9811-1605b0b4feb7 admin admin] PATH_INFO: `/v3/projects/75ffe18832094f0bae945ae881e0136e/users/d0fdf6eee08f4ae48deda78cd4b52b51/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.326323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbc943-0c94-4eb3-9811-1605b0b4feb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ffe18832094f0bae945ae881e0136e, user_id=d0fdf6eee08f4ae48deda78cd4b52b51, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:42.344809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbc943-0c94-4eb3-9811-1605b0b4feb7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:42.358303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9fee7cd4-d734-4470-844d-85cc94194d19 None None] MFA Rules not processed for user `75ef0e5dbd584576821298a35f1f5b05`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:42.361955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9fee7cd4-d734-4470-844d-85cc94194d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:42.380342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 980/1939] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:42 2022] PUT /identity/v3/projects/75ffe18832094f0bae945ae881e0136e/users/d0fdf6eee08f4ae48deda78cd4b52b51/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:42.383391 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbc943-0c94-4eb3-9811-1605b0b4feb7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:42.478270 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbc943-0c94-4eb3-9811-1605b0b4feb7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:42.486121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45e232-d32f-45c9-9c64-5096d0d00ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:42.487511 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45e232-d32f-45c9-9c64-5096d0d00ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.487869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45e232-d32f-45c9-9c64-5096d0d00ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.488235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45e232-d32f-45c9-9c64-5096d0d00ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.488964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45e232-d32f-45c9-9c64-5096d0d00ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:42.491122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45e232-d32f-45c9-9c64-5096d0d00ba3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:42.512661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 960/1940] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:42.526110 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c46e0-8b82-49d5-9c29-5795872716f0 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.526353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c46e0-8b82-49d5-9c29-5795872716f0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.528944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c46e0-8b82-49d5-9c29-5795872716f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.550785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 981/1941] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:42 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:42.550785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45e232-d32f-45c9-9c64-5096d0d00ba3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:42.571862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-580c46e0-8b82-49d5-9c29-5795872716f0 None None] MFA Rules not processed for user `d0fdf6eee08f4ae48deda78cd4b52b51`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:42.573251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-580c46e0-8b82-49d5-9c29-5795872716f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:42.597886 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:42.621699 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:42.696333 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45e232-d32f-45c9-9c64-5096d0d00ba3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:42.699143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 961/1942] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:42.699708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c46e0-8b82-49d5-9c29-5795872716f0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:42.712601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14310df-6cb0-4c29-8ae3-d1d2b112916f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:42.713721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14310df-6cb0-4c29-8ae3-d1d2b112916f service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.714007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14310df-6cb0-4c29-8ae3-d1d2b112916f service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.714283 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14310df-6cb0-4c29-8ae3-d1d2b112916f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.846689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c46e0-8b82-49d5-9c29-5795872716f0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:42.854205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14310df-6cb0-4c29-8ae3-d1d2b112916f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:42.857512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14310df-6cb0-4c29-8ae3-d1d2b112916f service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:42.861208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54317b2-2071-41a3-8f25-65da17f71634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:42.863513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54317b2-2071-41a3-8f25-65da17f71634 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.865437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54317b2-2071-41a3-8f25-65da17f71634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.866471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54317b2-2071-41a3-8f25-65da17f71634 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.868824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54317b2-2071-41a3-8f25-65da17f71634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:42.872747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54317b2-2071-41a3-8f25-65da17f71634 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:42.887234 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 982/1943] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:42.917834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 962/1944] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:42 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:42.922337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14310df-6cb0-4c29-8ae3-d1d2b112916f service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:42.940341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14310df-6cb0-4c29-8ae3-d1d2b112916f service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:42.957639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d718a7c-cf04-4ba0-bb8c-91744f604093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:42.958427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d718a7c-cf04-4ba0-bb8c-91744f604093 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.958868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d718a7c-cf04-4ba0-bb8c-91744f604093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.959374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d718a7c-cf04-4ba0-bb8c-91744f604093 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.960561 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d718a7c-cf04-4ba0-bb8c-91744f604093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:42.962105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d718a7c-cf04-4ba0-bb8c-91744f604093 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:42.982986 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 983/1945] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:42.988078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54317b2-2071-41a3-8f25-65da17f71634 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:43.003353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d718a7c-cf04-4ba0-bb8c-91744f604093 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:43.005861 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6d718a7c-cf04-4ba0-bb8c-91744f604093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:43.007032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54317b2-2071-41a3-8f25-65da17f71634 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:43.015966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5c98a-d17f-4879-8d0f-0cbc61d5d163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:43.016734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5c98a-d17f-4879-8d0f-0cbc61d5d163 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:43.017018 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5c98a-d17f-4879-8d0f-0cbc61d5d163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:43.017310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5c98a-d17f-4879-8d0f-0cbc61d5d163 admin admin] PATH_INFO: `/v3/projects/2468b2485a794051a983c7973d6e8915/users/9cfa653d64c547458cdeef2cadf5bbeb/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:43.024246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d718a7c-cf04-4ba0-bb8c-91744f604093 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:43.033025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:43.033922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:43.034432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:43.034912 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] PATH_INFO: `/v3/users/212ad17a91dc4a60a5c5ed1f05e0e123` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:43.036416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=212ad17a91dc4a60a5c5ed1f05e0e123)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:43.039199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:43.039934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5c98a-d17f-4879-8d0f-0cbc61d5d163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2468b2485a794051a983c7973d6e8915, user_id=9cfa653d64c547458cdeef2cadf5bbeb, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:43.043964 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5c98a-d17f-4879-8d0f-0cbc61d5d163 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:43.069772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 963/1946] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:42 2022] PUT /identity/v3/projects/2468b2485a794051a983c7973d6e8915/users/9cfa653d64c547458cdeef2cadf5bbeb/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:43.069772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5c98a-d17f-4879-8d0f-0cbc61d5d163 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:43.072623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d2c5c98a-d17f-4879-8d0f-0cbc61d5d163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:43.147720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212ad17a91dc4a60a5c5ed1f05e0e123'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:43.148862 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212ad17a91dc4a60a5c5ed1f05e0e123'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:43.149214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5c98a-d17f-4879-8d0f-0cbc61d5d163 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:43.149747 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212ad17a91dc4a60a5c5ed1f05e0e123'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:43.153459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299675c2-f3ae-4f0f-a3c1-4b2be529099f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:43.154186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299675c2-f3ae-4f0f-a3c1-4b2be529099f service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:43.154511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299675c2-f3ae-4f0f-a3c1-4b2be529099f service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:43.154888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299675c2-f3ae-4f0f-a3c1-4b2be529099f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:43.157696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-299675c2-f3ae-4f0f-a3c1-4b2be529099f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:43.159809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212ad17a91dc4a60a5c5ed1f05e0e123'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:43.170475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212ad17a91dc4a60a5c5ed1f05e0e123'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:43.221471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:43.226882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 984/1947] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:43 2022] DELETE /identity/v3/users/212ad17a91dc4a60a5c5ed1f05e0e123 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:43.226882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:43.228577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:43.287926 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299675c2-f3ae-4f0f-a3c1-4b2be529099f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:43.289590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299675c2-f3ae-4f0f-a3c1-4b2be529099f service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:43.316188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:43.322064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 964/1948] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:43.326491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde1b7e-9193-4842-a295-e88e7d3ac78a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:43.326785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde1b7e-9193-4842-a295-e88e7d3ac78a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:43.327065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde1b7e-9193-4842-a295-e88e7d3ac78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:43.382082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bbde1b7e-9193-4842-a295-e88e7d3ac78a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:43.387391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bbde1b7e-9193-4842-a295-e88e7d3ac78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:43.407853 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db087039-8e48-487b-8a8a-727b49a8a513 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:43.417321 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d631da-ae2c-46d3-93ec-35a99cb21645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:43.418058 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d631da-ae2c-46d3-93ec-35a99cb21645 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:43.418436 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d631da-ae2c-46d3-93ec-35a99cb21645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:43.418806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d631da-ae2c-46d3-93ec-35a99cb21645 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:43.422510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d631da-ae2c-46d3-93ec-35a99cb21645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:43.422510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d631da-ae2c-46d3-93ec-35a99cb21645 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:43.424346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 965/1949] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:43.432646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde1b7e-9193-4842-a295-e88e7d3ac78a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:43.434241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bbde1b7e-9193-4842-a295-e88e7d3ac78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:43.434514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 985/1950] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:43.440873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d631da-ae2c-46d3-93ec-35a99cb21645 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:43.472462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d631da-ae2c-46d3-93ec-35a99cb21645 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:43.479137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339ffb8-690b-457d-a2ac-10f11661673d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:43.480098 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339ffb8-690b-457d-a2ac-10f11661673d admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:43.480467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339ffb8-690b-457d-a2ac-10f11661673d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:43.480751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339ffb8-690b-457d-a2ac-10f11661673d admin admin] PATH_INFO: `/v3/projects/2468b2485a794051a983c7973d6e8915/users/9cfa653d64c547458cdeef2cadf5bbeb/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:43.484429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339ffb8-690b-457d-a2ac-10f11661673d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2468b2485a794051a983c7973d6e8915, user_id=9cfa653d64c547458cdeef2cadf5bbeb, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:43.488828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339ffb8-690b-457d-a2ac-10f11661673d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:43.516006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 986/1951] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:43 2022] PUT /identity/v3/projects/2468b2485a794051a983c7973d6e8915/users/9cfa653d64c547458cdeef2cadf5bbeb/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:43.521015 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339ffb8-690b-457d-a2ac-10f11661673d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:43.529705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0339ffb8-690b-457d-a2ac-10f11661673d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:43.575809 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde1b7e-9193-4842-a295-e88e7d3ac78a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:43.613839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339ffb8-690b-457d-a2ac-10f11661673d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:43.617520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c02182-5b9a-4133-970f-b1c6ba22a2de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:43.618465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02182-5b9a-4133-970f-b1c6ba22a2de service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:43.618758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02182-5b9a-4133-970f-b1c6ba22a2de service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:43.619039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02182-5b9a-4133-970f-b1c6ba22a2de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:43.622595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-60c02182-5b9a-4133-970f-b1c6ba22a2de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:43.631335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949a862-1cdb-4475-9a6a-9858a0288361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:43.632230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949a862-1cdb-4475-9a6a-9858a0288361 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:43.632738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949a862-1cdb-4475-9a6a-9858a0288361 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:43.633262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949a862-1cdb-4475-9a6a-9858a0288361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:43.696912 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c02182-5b9a-4133-970f-b1c6ba22a2de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:43.698579 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c02182-5b9a-4133-970f-b1c6ba22a2de service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:43.700291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949a862-1cdb-4475-9a6a-9858a0288361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:43.701978 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949a862-1cdb-4475-9a6a-9858a0288361 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:43.710734 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 987/1952] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:43.712038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac2565-ac71-40ce-81b4-027ed0afc101 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:43.712465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac2565-ac71-40ce-81b4-027ed0afc101 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:43.712786 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac2565-ac71-40ce-81b4-027ed0afc101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:43.724160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 966/1953] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:43.725019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949a862-1cdb-4475-9a6a-9858a0288361 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:43.726549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a949a862-1cdb-4475-9a6a-9858a0288361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:43.746869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f6ac2565-ac71-40ce-81b4-027ed0afc101 None None] MFA Rules not processed for user `9cfa653d64c547458cdeef2cadf5bbeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:43.748440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f6ac2565-ac71-40ce-81b4-027ed0afc101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:43.750974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949a862-1cdb-4475-9a6a-9858a0288361 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:43.755243 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13bb07-07f5-4ee4-a7f9-230a0e16781b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:43.755972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13bb07-07f5-4ee4-a7f9-230a0e16781b service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:43.756386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13bb07-07f5-4ee4-a7f9-230a0e16781b service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:43.756726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13bb07-07f5-4ee4-a7f9-230a0e16781b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:43.759041 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-da13bb07-07f5-4ee4-a7f9-230a0e16781b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:43.827354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:43.873596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13bb07-07f5-4ee4-a7f9-230a0e16781b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:43.875672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 988/1954] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:43.876454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13bb07-07f5-4ee4-a7f9-230a0e16781b service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:43.891829 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac2565-ac71-40ce-81b4-027ed0afc101 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:43.904488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac2565-ac71-40ce-81b4-027ed0afc101 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:43.906322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 967/1955] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:43.910334 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21558ea-4062-44bc-a864-b2938ede8ede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:43.911005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21558ea-4062-44bc-a864-b2938ede8ede service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:43.911350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21558ea-4062-44bc-a864-b2938ede8ede service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:43.911628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21558ea-4062-44bc-a864-b2938ede8ede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:43.918535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:43.924949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21558ea-4062-44bc-a864-b2938ede8ede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:43.927128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21558ea-4062-44bc-a864-b2938ede8ede service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:43.950200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 989/1956] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:44.072840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13bb07-07f5-4ee4-a7f9-230a0e16781b service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:44.086789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13bb07-07f5-4ee4-a7f9-230a0e16781b service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:44.090692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fd833-a929-4921-af1b-29f6d1f98c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:44.091392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fd833-a929-4921-af1b-29f6d1f98c8d service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:44.091635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fd833-a929-4921-af1b-29f6d1f98c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:44.091914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fd833-a929-4921-af1b-29f6d1f98c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:44.093840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-459fd833-a929-4921-af1b-29f6d1f98c8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:44.141883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21558ea-4062-44bc-a864-b2938ede8ede service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:44.143832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f21558ea-4062-44bc-a864-b2938ede8ede service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:44.163672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21558ea-4062-44bc-a864-b2938ede8ede service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:44.170053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fd833-a929-4921-af1b-29f6d1f98c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:44.172158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fd833-a929-4921-af1b-29f6d1f98c8d service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:44.177990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:44.178662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:44.178946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:44.179224 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] PATH_INFO: `/v3/projects/9f8ad395e42a42598af71dfcb251d76f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:44.180664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f8ad395e42a42598af71dfcb251d76f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:44.182808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:44.184398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 968/1957] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:44.243547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f8ad395e42a42598af71dfcb251d76f', 'request_id': 'req-f37ecda7-c83f-4252-81df-3d4cd1f95573'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:44.272829 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f8ad395e42a42598af71dfcb251d76f', 'request_id': 'req-f37ecda7-c83f-4252-81df-3d4cd1f95573'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:44.281504 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f8ad395e42a42598af71dfcb251d76f', 'request_id': 'req-f37ecda7-c83f-4252-81df-3d4cd1f95573'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:44.303157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] The token cache is being invalidate because project 9f8ad395e42a42598af71dfcb251d76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:44.303157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:44.303786 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:44.310827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fd833-a929-4921-af1b-29f6d1f98c8d service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:44.312771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 990/1958] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:44 2022] DELETE /identity/v3/projects/9f8ad395e42a42598af71dfcb251d76f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:44.324686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fd833-a929-4921-af1b-29f6d1f98c8d service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:44.388196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a83ffe-206b-4a5a-babb-1bf8d248d7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:44.388849 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a83ffe-206b-4a5a-babb-1bf8d248d7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:44.389143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a83ffe-206b-4a5a-babb-1bf8d248d7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:44.389386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a83ffe-206b-4a5a-babb-1bf8d248d7f4 admin admin] PATH_INFO: `/v3/users/1ec901d609804e9fa8be4bbab3b03d60` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:44.391114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a83ffe-206b-4a5a-babb-1bf8d248d7f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ec901d609804e9fa8be4bbab3b03d60)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:44.393136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a83ffe-206b-4a5a-babb-1bf8d248d7f4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:44.444146 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-58a83ffe-206b-4a5a-babb-1bf8d248d7f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ec901d609804e9fa8be4bbab3b03d60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:44.444819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-58a83ffe-206b-4a5a-babb-1bf8d248d7f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ec901d609804e9fa8be4bbab3b03d60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:44.445458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-58a83ffe-206b-4a5a-babb-1bf8d248d7f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ec901d609804e9fa8be4bbab3b03d60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:44.454053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-58a83ffe-206b-4a5a-babb-1bf8d248d7f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ec901d609804e9fa8be4bbab3b03d60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:44.461785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-58a83ffe-206b-4a5a-babb-1bf8d248d7f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ec901d609804e9fa8be4bbab3b03d60'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:44.470205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:44.491200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 969/1959] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:44 2022] DELETE /identity/v3/users/1ec901d609804e9fa8be4bbab3b03d60 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:45.037372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:45.039059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:45.061912 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ecda7-c83f-4252-81df-3d4cd1f95573 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:45.129125 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:45.129891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:45.130241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:45.130591 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] PATH_INFO: `/v3/projects/b8ff204c4e7941bf96c14d7db06877d0` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:45.131864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ff204c4e7941bf96c14d7db06877d0)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:45.134012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:45.195375 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ff204c4e7941bf96c14d7db06877d0', 'request_id': 'req-4db1e42c-adc3-46db-9658-130d3ffc93cd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:45.202817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ff204c4e7941bf96c14d7db06877d0', 'request_id': 'req-4db1e42c-adc3-46db-9658-130d3ffc93cd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:45.209162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ff204c4e7941bf96c14d7db06877d0', 'request_id': 'req-4db1e42c-adc3-46db-9658-130d3ffc93cd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:45.224904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] The token cache is being invalidate because project b8ff204c4e7941bf96c14d7db06877d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:45.225195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:45.227047 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4db1e42c-adc3-46db-9658-130d3ffc93cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:45.229205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 991/1960] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:45 2022] DELETE /identity/v3/projects/b8ff204c4e7941bf96c14d7db06877d0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:45.233982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734814cc-d8c0-4d2d-9162-4d8963979727 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:45.235017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734814cc-d8c0-4d2d-9162-4d8963979727 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:45.236474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734814cc-d8c0-4d2d-9162-4d8963979727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:45.261180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-734814cc-d8c0-4d2d-9162-4d8963979727 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:45.262783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-734814cc-d8c0-4d2d-9162-4d8963979727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:45.347851 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 970/1961] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:45.355512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba75e49-5977-43e5-a596-8c4819604d69 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:45.355800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba75e49-5977-43e5-a596-8c4819604d69 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:45.356106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba75e49-5977-43e5-a596-8c4819604d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:45.371919 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:45.378877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-aba75e49-5977-43e5-a596-8c4819604d69 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:45.381183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-aba75e49-5977-43e5-a596-8c4819604d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:45.405646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 992/1962] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:45.409858 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734814cc-d8c0-4d2d-9162-4d8963979727 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:45.421765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734814cc-d8c0-4d2d-9162-4d8963979727 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:45.427151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922df8d6-84cb-4d95-9d79-2df2e88e4dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:45.427885 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922df8d6-84cb-4d95-9d79-2df2e88e4dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:45.428195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922df8d6-84cb-4d95-9d79-2df2e88e4dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:45.428467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922df8d6-84cb-4d95-9d79-2df2e88e4dd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:45.429123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922df8d6-84cb-4d95-9d79-2df2e88e4dd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:45.429426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922df8d6-84cb-4d95-9d79-2df2e88e4dd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:45.431201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922df8d6-84cb-4d95-9d79-2df2e88e4dd9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:45.445639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 971/1963] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:45.450225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba75e49-5977-43e5-a596-8c4819604d69 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:45.460890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba75e49-5977-43e5-a596-8c4819604d69 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:45.469169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46878f4-9123-4d9c-94aa-5efc7eddfc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:45.469731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46878f4-9123-4d9c-94aa-5efc7eddfc27 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:45.470059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46878f4-9123-4d9c-94aa-5efc7eddfc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:45.470378 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46878f4-9123-4d9c-94aa-5efc7eddfc27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:45.471110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46878f4-9123-4d9c-94aa-5efc7eddfc27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:45.473074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46878f4-9123-4d9c-94aa-5efc7eddfc27 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:45.507413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 993/1964] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:45 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:45.511915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922df8d6-84cb-4d95-9d79-2df2e88e4dd9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:45.531015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922df8d6-84cb-4d95-9d79-2df2e88e4dd9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:45.637435 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e52f35-781b-44e8-8447-2f4473bfe9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:45.638339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e52f35-781b-44e8-8447-2f4473bfe9f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:45.638816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e52f35-781b-44e8-8447-2f4473bfe9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:45.639165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e52f35-781b-44e8-8447-2f4473bfe9f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:45.640125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e52f35-781b-44e8-8447-2f4473bfe9f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:45.643393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e52f35-781b-44e8-8447-2f4473bfe9f0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:45.678783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 972/1965] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:45 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:45.683386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46878f4-9123-4d9c-94aa-5efc7eddfc27 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:45.708768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46878f4-9123-4d9c-94aa-5efc7eddfc27 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:45.713771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d8fbe-316c-4e15-8eb4-eef5beb42683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:45.714475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d8fbe-316c-4e15-8eb4-eef5beb42683 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:45.714818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d8fbe-316c-4e15-8eb4-eef5beb42683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:45.715153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d8fbe-316c-4e15-8eb4-eef5beb42683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:45.715899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d8fbe-316c-4e15-8eb4-eef5beb42683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:45.717649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d8fbe-316c-4e15-8eb4-eef5beb42683 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:45.731568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 994/1966] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:45.733064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e52f35-781b-44e8-8447-2f4473bfe9f0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:45.743229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e52f35-781b-44e8-8447-2f4473bfe9f0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:45.749825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f2e06-40d2-4e66-80bb-beda531e15ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:45.750474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f2e06-40d2-4e66-80bb-beda531e15ee admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:45.750767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f2e06-40d2-4e66-80bb-beda531e15ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:45.750998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f2e06-40d2-4e66-80bb-beda531e15ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:45.751645 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f2e06-40d2-4e66-80bb-beda531e15ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:45.753393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f2e06-40d2-4e66-80bb-beda531e15ee admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:45.776093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 973/1967] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:45.779963 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d8fbe-316c-4e15-8eb4-eef5beb42683 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:45.796092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d8fbe-316c-4e15-8eb4-eef5beb42683 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:45.800883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaff4e8-ba06-4cdc-a5c1-fb7e43e9e6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:45.801538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaff4e8-ba06-4cdc-a5c1-fb7e43e9e6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:45.801819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaff4e8-ba06-4cdc-a5c1-fb7e43e9e6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:45.802096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaff4e8-ba06-4cdc-a5c1-fb7e43e9e6fe admin admin] PATH_INFO: `/v3/projects/350e782609b74b90a9b6b68937d054fc/users/5a63449fea2d4db68ec3e5476d03b490/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:45.819142 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaff4e8-ba06-4cdc-a5c1-fb7e43e9e6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=350e782609b74b90a9b6b68937d054fc, user_id=5a63449fea2d4db68ec3e5476d03b490, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:45.822102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaff4e8-ba06-4cdc-a5c1-fb7e43e9e6fe admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:45.836310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 995/1968] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:45 2022] PUT /identity/v3/projects/350e782609b74b90a9b6b68937d054fc/users/5a63449fea2d4db68ec3e5476d03b490/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:45.847704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3ce0f-ddc2-4156-8486-28c4a79e1c80 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:45.847991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3ce0f-ddc2-4156-8486-28c4a79e1c80 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:45.848413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3ce0f-ddc2-4156-8486-28c4a79e1c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:45.882581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7dd3ce0f-ddc2-4156-8486-28c4a79e1c80 None None] MFA Rules not processed for user `5a63449fea2d4db68ec3e5476d03b490`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:45.883803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7dd3ce0f-ddc2-4156-8486-28c4a79e1c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:45.926566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:45.953057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 974/1969] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:45.957535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaff4e8-ba06-4cdc-a5c1-fb7e43e9e6fe admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:46.021504 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaff4e8-ba06-4cdc-a5c1-fb7e43e9e6fe admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:46.026419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee79c3-a907-4dfe-a18a-09b09b6ba338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:46.026994 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee79c3-a907-4dfe-a18a-09b09b6ba338 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:46.027279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee79c3-a907-4dfe-a18a-09b09b6ba338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:46.027554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee79c3-a907-4dfe-a18a-09b09b6ba338 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:46.028274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee79c3-a907-4dfe-a18a-09b09b6ba338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:46.030435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee79c3-a907-4dfe-a18a-09b09b6ba338 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:46.060816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 996/1970] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:45 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:46.062705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3ce0f-ddc2-4156-8486-28c4a79e1c80 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:46.073497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3ce0f-ddc2-4156-8486-28c4a79e1c80 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:46.135992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dcf7e-c545-4b29-bf60-b57038723406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:46.136860 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dcf7e-c545-4b29-bf60-b57038723406 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:46.137145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dcf7e-c545-4b29-bf60-b57038723406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:46.137421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dcf7e-c545-4b29-bf60-b57038723406 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:46.138056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3dcf7e-c545-4b29-bf60-b57038723406 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:46.139983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3dcf7e-c545-4b29-bf60-b57038723406 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:46.161676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 975/1971] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:46 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:46.165867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee79c3-a907-4dfe-a18a-09b09b6ba338 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:46.176652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee79c3-a907-4dfe-a18a-09b09b6ba338 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:46.181643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff4da2-d2e1-4964-91df-0a0569eeea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:46.182252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff4da2-d2e1-4964-91df-0a0569eeea5f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:46.182536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff4da2-d2e1-4964-91df-0a0569eeea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:46.182937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff4da2-d2e1-4964-91df-0a0569eeea5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:46.183532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff4da2-d2e1-4964-91df-0a0569eeea5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:46.185184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff4da2-d2e1-4964-91df-0a0569eeea5f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:46.196867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 997/1972] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:46.204565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dcf7e-c545-4b29-bf60-b57038723406 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:46.214908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dcf7e-c545-4b29-bf60-b57038723406 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:46.219515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe215663-c480-4ad1-908d-4f4af270ddd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:46.220194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe215663-c480-4ad1-908d-4f4af270ddd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:46.220479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe215663-c480-4ad1-908d-4f4af270ddd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:46.220759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe215663-c480-4ad1-908d-4f4af270ddd8 admin admin] PATH_INFO: `/v3/projects/584d59865a144b9f843643f070089093/users/9e699cf246924d6e9bccab4bff79e29c/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:46.236061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe215663-c480-4ad1-908d-4f4af270ddd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=584d59865a144b9f843643f070089093, user_id=9e699cf246924d6e9bccab4bff79e29c, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:46.238736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe215663-c480-4ad1-908d-4f4af270ddd8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:46.252878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 976/1973] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:46 2022] PUT /identity/v3/projects/584d59865a144b9f843643f070089093/users/9e699cf246924d6e9bccab4bff79e29c/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:46.256763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff4da2-d2e1-4964-91df-0a0569eeea5f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:46.266579 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff4da2-d2e1-4964-91df-0a0569eeea5f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:46.323028 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07748b-6231-4101-8f11-e6d7e9e242bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:46.323626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07748b-6231-4101-8f11-e6d7e9e242bc admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:46.323904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07748b-6231-4101-8f11-e6d7e9e242bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:46.324275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07748b-6231-4101-8f11-e6d7e9e242bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:46.324918 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07748b-6231-4101-8f11-e6d7e9e242bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:46.326510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07748b-6231-4101-8f11-e6d7e9e242bc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:46.336204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 998/1974] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:46.340482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe215663-c480-4ad1-908d-4f4af270ddd8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:46.350889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe215663-c480-4ad1-908d-4f4af270ddd8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:46.355812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c786e49-f515-4055-9781-ab160b154d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:46.356547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c786e49-f515-4055-9781-ab160b154d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:46.356823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c786e49-f515-4055-9781-ab160b154d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:46.357095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c786e49-f515-4055-9781-ab160b154d43 admin admin] PATH_INFO: `/v3/projects/584d59865a144b9f843643f070089093/users/9e699cf246924d6e9bccab4bff79e29c/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:46.359734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c786e49-f515-4055-9781-ab160b154d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=584d59865a144b9f843643f070089093, user_id=9e699cf246924d6e9bccab4bff79e29c, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:46.362623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c786e49-f515-4055-9781-ab160b154d43 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:46.373307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 977/1975] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:46 2022] PUT /identity/v3/projects/584d59865a144b9f843643f070089093/users/9e699cf246924d6e9bccab4bff79e29c/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:46.384785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89faad-f91a-497d-92a4-162120f6947b None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:46.385072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89faad-f91a-497d-92a4-162120f6947b None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:46.385350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89faad-f91a-497d-92a4-162120f6947b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:46.409461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:46.412630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ae89faad-f91a-497d-92a4-162120f6947b None None] MFA Rules not processed for user `9e699cf246924d6e9bccab4bff79e29c`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:46.414552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ae89faad-f91a-497d-92a4-162120f6947b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:46.484201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 999/1976] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:46 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:46.495162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c786e49-f515-4055-9781-ab160b154d43 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:46.496971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7c786e49-f515-4055-9781-ab160b154d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:46.554444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c786e49-f515-4055-9781-ab160b154d43 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:46.558456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e7e5c-655d-4bde-81b0-6399c685661e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:46.559159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e7e5c-655d-4bde-81b0-6399c685661e service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:46.559323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e7e5c-655d-4bde-81b0-6399c685661e service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:46.559596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e7e5c-655d-4bde-81b0-6399c685661e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:46.569338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e7e5c-655d-4bde-81b0-6399c685661e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:46.570763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e7e5c-655d-4bde-81b0-6399c685661e service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:46.580652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 978/1977] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:46.991585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89faad-f91a-497d-92a4-162120f6947b None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:47.054579 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89faad-f91a-497d-92a4-162120f6947b None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:47.059418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:47.059939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:47.060290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:47.060523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] PATH_INFO: `/v3/users/5a63449fea2d4db68ec3e5476d03b490` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:47.061658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a63449fea2d4db68ec3e5476d03b490)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:47.063654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:47.114351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a63449fea2d4db68ec3e5476d03b490'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.115011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a63449fea2d4db68ec3e5476d03b490'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.115612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a63449fea2d4db68ec3e5476d03b490'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.123452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a63449fea2d4db68ec3e5476d03b490'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.131083 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a63449fea2d4db68ec3e5476d03b490'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.148987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1000/1978] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:46 2022] DELETE /identity/v3/users/5a63449fea2d4db68ec3e5476d03b490 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:47.153675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239bb91-034e-40c1-9146-5c712dd8ec6e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:47.154129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239bb91-034e-40c1-9146-5c712dd8ec6e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:47.154473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239bb91-034e-40c1-9146-5c712dd8ec6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:47.175339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8239bb91-034e-40c1-9146-5c712dd8ec6e None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:47.176584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8239bb91-034e-40c1-9146-5c712dd8ec6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:47.213032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:47.243104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 979/1979] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:47.253358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:47.254791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:47.276598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:47.320670 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9a240-c387-4be3-908a-20d0faeca735 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:47.324782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762b69a-89b6-4bd4-af02-70afd4597ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:47.325463 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762b69a-89b6-4bd4-af02-70afd4597ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:47.325834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762b69a-89b6-4bd4-af02-70afd4597ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:47.326158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762b69a-89b6-4bd4-af02-70afd4597ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:47.335808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762b69a-89b6-4bd4-af02-70afd4597ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:47.337363 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762b69a-89b6-4bd4-af02-70afd4597ef2 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:47.348014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1001/1980] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:47.664714 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239bb91-034e-40c1-9146-5c712dd8ec6e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:47.666235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8239bb91-034e-40c1-9146-5c712dd8ec6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:47.682150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239bb91-034e-40c1-9146-5c712dd8ec6e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:47.686858 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:47.687504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:47.687784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:47.688079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] PATH_INFO: `/v3/projects/350e782609b74b90a9b6b68937d054fc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:47.689236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=350e782609b74b90a9b6b68937d054fc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:47.691183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:47.751927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350e782609b74b90a9b6b68937d054fc', 'request_id': 'req-218081a0-2054-43a5-ab66-d0b5b123ae29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.762087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350e782609b74b90a9b6b68937d054fc', 'request_id': 'req-218081a0-2054-43a5-ab66-d0b5b123ae29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.784653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350e782609b74b90a9b6b68937d054fc', 'request_id': 'req-218081a0-2054-43a5-ab66-d0b5b123ae29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.806538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] The token cache is being invalidate because project 350e782609b74b90a9b6b68937d054fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:47.806900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.807577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.809342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 980/1981] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:47 2022] DELETE /identity/v3/projects/350e782609b74b90a9b6b68937d054fc => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:47.813591 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762b69a-89b6-4bd4-af02-70afd4597ef2 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:47.823814 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762b69a-89b6-4bd4-af02-70afd4597ef2 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:47.883303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bd00a-d58c-45c7-8de4-7724837bf77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:47.883980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bd00a-d58c-45c7-8de4-7724837bf77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:47.884327 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bd00a-d58c-45c7-8de4-7724837bf77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:47.884635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bd00a-d58c-45c7-8de4-7724837bf77c admin admin] PATH_INFO: `/v3/users/9e699cf246924d6e9bccab4bff79e29c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:47.885944 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bd00a-d58c-45c7-8de4-7724837bf77c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e699cf246924d6e9bccab4bff79e29c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:47.888023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bd00a-d58c-45c7-8de4-7724837bf77c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:47.941482 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7d5bd00a-d58c-45c7-8de4-7724837bf77c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e699cf246924d6e9bccab4bff79e29c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.942144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7d5bd00a-d58c-45c7-8de4-7724837bf77c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e699cf246924d6e9bccab4bff79e29c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.942769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7d5bd00a-d58c-45c7-8de4-7724837bf77c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e699cf246924d6e9bccab4bff79e29c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.957217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7d5bd00a-d58c-45c7-8de4-7724837bf77c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e699cf246924d6e9bccab4bff79e29c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.966206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-7d5bd00a-d58c-45c7-8de4-7724837bf77c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e699cf246924d6e9bccab4bff79e29c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.988291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1002/1982] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:47 2022] DELETE /identity/v3/users/9e699cf246924d6e9bccab4bff79e29c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:48.295171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.298480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:48.328446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218081a0-2054-43a5-ab66-d0b5b123ae29 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.399808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.400568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.400920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.401268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] PATH_INFO: `/v3/projects/584d59865a144b9f843643f070089093` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.403073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=584d59865a144b9f843643f070089093)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.405336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.480963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '584d59865a144b9f843643f070089093', 'request_id': 'req-88ba335c-5e6e-4e00-9805-9e0683256d6f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:48.488239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '584d59865a144b9f843643f070089093', 'request_id': 'req-88ba335c-5e6e-4e00-9805-9e0683256d6f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:48.494404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '584d59865a144b9f843643f070089093', 'request_id': 'req-88ba335c-5e6e-4e00-9805-9e0683256d6f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:48.496252 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:48.512038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] The token cache is being invalidate because project 584d59865a144b9f843643f070089093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:48.513009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:48.514084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-88ba335c-5e6e-4e00-9805-9e0683256d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:48.515926 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 981/1983] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:48 2022] DELETE /identity/v3/projects/584d59865a144b9f843643f070089093 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:48.520973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405eaaa-9423-46c7-bff4-5975fab52818 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.521427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405eaaa-9423-46c7-bff4-5975fab52818 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.521659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405eaaa-9423-46c7-bff4-5975fab52818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.545102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e405eaaa-9423-46c7-bff4-5975fab52818 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:48.546390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e405eaaa-9423-46c7-bff4-5975fab52818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:48.583338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:48.612431 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1003/1984] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:48.621040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcac7a-efe9-4e16-abcf-7a5b280ef0a7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.621321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcac7a-efe9-4e16-abcf-7a5b280ef0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.621634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcac7a-efe9-4e16-abcf-7a5b280ef0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.641353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-28bcac7a-efe9-4e16-abcf-7a5b280ef0a7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:48.642549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-28bcac7a-efe9-4e16-abcf-7a5b280ef0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:48.650770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 982/1985] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:48.654845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405eaaa-9423-46c7-bff4-5975fab52818 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.666635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405eaaa-9423-46c7-bff4-5975fab52818 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.671558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab910c12-5774-4087-993c-a8571fab95ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.672195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab910c12-5774-4087-993c-a8571fab95ef admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.672481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab910c12-5774-4087-993c-a8571fab95ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.672756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab910c12-5774-4087-993c-a8571fab95ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.673471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab910c12-5774-4087-993c-a8571fab95ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:48.673777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab910c12-5774-4087-993c-a8571fab95ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.675539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab910c12-5774-4087-993c-a8571fab95ef admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.689628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1004/1986] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:48.693771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcac7a-efe9-4e16-abcf-7a5b280ef0a7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.703937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcac7a-efe9-4e16-abcf-7a5b280ef0a7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.708768 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e55e7-c549-4a2d-a165-28b409a10cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.709365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e55e7-c549-4a2d-a165-28b409a10cba admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.709832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e55e7-c549-4a2d-a165-28b409a10cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.710105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e55e7-c549-4a2d-a165-28b409a10cba admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.711248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e55e7-c549-4a2d-a165-28b409a10cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.713212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e55e7-c549-4a2d-a165-28b409a10cba admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.747665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 983/1987] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:48 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:48.749397 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab910c12-5774-4087-993c-a8571fab95ef admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.763021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab910c12-5774-4087-993c-a8571fab95ef admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.776150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e55e7-c549-4a2d-a165-28b409a10cba admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.777877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c27e55e7-c549-4a2d-a165-28b409a10cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:48.849522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017932a-35fd-4486-b0dd-535a6bf2ca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.851282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017932a-35fd-4486-b0dd-535a6bf2ca40 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.854423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017932a-35fd-4486-b0dd-535a6bf2ca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.854423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017932a-35fd-4486-b0dd-535a6bf2ca40 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.854423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017932a-35fd-4486-b0dd-535a6bf2ca40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.854676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e55e7-c549-4a2d-a165-28b409a10cba admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.855253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017932a-35fd-4486-b0dd-535a6bf2ca40 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.861097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc18a77-7347-4b48-ad35-28c3d9ac7be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.861820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc18a77-7347-4b48-ad35-28c3d9ac7be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.862106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc18a77-7347-4b48-ad35-28c3d9ac7be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.862381 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc18a77-7347-4b48-ad35-28c3d9ac7be3 admin admin] PATH_INFO: `/v3/users/1bffd6da32a94611ace43d6eada4cae8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.863720 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc18a77-7347-4b48-ad35-28c3d9ac7be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bffd6da32a94611ace43d6eada4cae8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.866126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc18a77-7347-4b48-ad35-28c3d9ac7be3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.887898 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1005/1988] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:48 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:48.894215 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017932a-35fd-4486-b0dd-535a6bf2ca40 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.912367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017932a-35fd-4486-b0dd-535a6bf2ca40 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.938649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64306af-450f-48ba-b653-0ade9ac672b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.939222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cc18a77-7347-4b48-ad35-28c3d9ac7be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bffd6da32a94611ace43d6eada4cae8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:48.940093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cc18a77-7347-4b48-ad35-28c3d9ac7be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bffd6da32a94611ace43d6eada4cae8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:48.940953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cc18a77-7347-4b48-ad35-28c3d9ac7be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bffd6da32a94611ace43d6eada4cae8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:48.941263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64306af-450f-48ba-b653-0ade9ac672b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.943848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64306af-450f-48ba-b653-0ade9ac672b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.944200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64306af-450f-48ba-b653-0ade9ac672b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.944915 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64306af-450f-48ba-b653-0ade9ac672b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.946640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64306af-450f-48ba-b653-0ade9ac672b1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.955131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cc18a77-7347-4b48-ad35-28c3d9ac7be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bffd6da32a94611ace43d6eada4cae8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:48.962537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1006/1989] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:48.967583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64306af-450f-48ba-b653-0ade9ac672b1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.976616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cc18a77-7347-4b48-ad35-28c3d9ac7be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bffd6da32a94611ace43d6eada4cae8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:48.978748 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a64306af-450f-48ba-b653-0ade9ac672b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:49.027641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64306af-450f-48ba-b653-0ade9ac672b1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:49.034127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02792b11-f936-4911-b51a-f486350b14b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:49.034771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02792b11-f936-4911-b51a-f486350b14b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.035049 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02792b11-f936-4911-b51a-f486350b14b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.035324 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02792b11-f936-4911-b51a-f486350b14b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.036129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02792b11-f936-4911-b51a-f486350b14b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:49.037730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02792b11-f936-4911-b51a-f486350b14b3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:49.041679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 984/1990] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:48 2022] DELETE /identity/v3/users/1bffd6da32a94611ace43d6eada4cae8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:49.051695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6306-6cbb-49b4-923c-1f2bd5ff91b9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.052073 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6306-6cbb-49b4-923c-1f2bd5ff91b9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.052334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6306-6cbb-49b4-923c-1f2bd5ff91b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.052479 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1007/1991] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:49.054097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02792b11-f936-4911-b51a-f486350b14b3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:49.057844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-02792b11-f936-4911-b51a-f486350b14b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:49.072258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02792b11-f936-4911-b51a-f486350b14b3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:49.087402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-480a6306-6cbb-49b4-923c-1f2bd5ff91b9 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:49.090618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-480a6306-6cbb-49b4-923c-1f2bd5ff91b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:49.154089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d4764-d55b-43f8-a163-31db87ae3752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:49.154848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d4764-d55b-43f8-a163-31db87ae3752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.155377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d4764-d55b-43f8-a163-31db87ae3752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.155737 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d4764-d55b-43f8-a163-31db87ae3752 admin admin] PATH_INFO: `/v3/users/72732b3547fa461ea5a0219dd7ea8537` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.157449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d4764-d55b-43f8-a163-31db87ae3752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72732b3547fa461ea5a0219dd7ea8537)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:49.159389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d4764-d55b-43f8-a163-31db87ae3752 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:49.190998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=3,keystone:DELETE=11,keystone:SELECT=86 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:49.231928 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:49.238883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 985/1992] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:49.238883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a6306-6cbb-49b4-923c-1f2bd5ff91b9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:49.251626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2c5d4764-d55b-43f8-a163-31db87ae3752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72732b3547fa461ea5a0219dd7ea8537'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:49.251814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a6306-6cbb-49b4-923c-1f2bd5ff91b9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:49.252888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2c5d4764-d55b-43f8-a163-31db87ae3752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72732b3547fa461ea5a0219dd7ea8537'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:49.254138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2c5d4764-d55b-43f8-a163-31db87ae3752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72732b3547fa461ea5a0219dd7ea8537'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:49.257939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3f5d6-524d-4ee7-9c93-1a4b77c0a00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:49.258855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3f5d6-524d-4ee7-9c93-1a4b77c0a00e admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.259154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3f5d6-524d-4ee7-9c93-1a4b77c0a00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.259500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3f5d6-524d-4ee7-9c93-1a4b77c0a00e admin admin] PATH_INFO: `/v3/projects/963cbe680f32424f8ba6b611e4150d50/users/aa6af53dd62248d09a8b8723df4e29f8/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.274919 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2c5d4764-d55b-43f8-a163-31db87ae3752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72732b3547fa461ea5a0219dd7ea8537'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:49.278184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3f5d6-524d-4ee7-9c93-1a4b77c0a00e admin admin] RBAC: Authorizing `identity:create_grant(project_id=963cbe680f32424f8ba6b611e4150d50, user_id=aa6af53dd62248d09a8b8723df4e29f8, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:49.282143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3f5d6-524d-4ee7-9c93-1a4b77c0a00e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:49.285277 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2c5d4764-d55b-43f8-a163-31db87ae3752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72732b3547fa461ea5a0219dd7ea8537'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:49.296677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 986/1993] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:49 2022] PUT /identity/v3/projects/963cbe680f32424f8ba6b611e4150d50/users/aa6af53dd62248d09a8b8723df4e29f8/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:49.297673 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3f5d6-524d-4ee7-9c93-1a4b77c0a00e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:49.299185 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-24f3f5d6-524d-4ee7-9c93-1a4b77c0a00e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:49.320089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1008/1994] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:49 2022] DELETE /identity/v3/users/72732b3547fa461ea5a0219dd7ea8537 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:49.322562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d7eab-d049-4663-b4e4-26ea4de7ab21 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.322867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d7eab-d049-4663-b4e4-26ea4de7ab21 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.323143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d7eab-d049-4663-b4e4-26ea4de7ab21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.357577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-174d7eab-d049-4663-b4e4-26ea4de7ab21 None None] MFA Rules not processed for user `aa6af53dd62248d09a8b8723df4e29f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:49.358809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-174d7eab-d049-4663-b4e4-26ea4de7ab21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:49.362427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3f5d6-524d-4ee7-9c93-1a4b77c0a00e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:49.412590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3c4ac-c53d-478d-be90-453b8dce9441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:49.413337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3c4ac-c53d-478d-be90-453b8dce9441 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.413756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3c4ac-c53d-478d-be90-453b8dce9441 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.414036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3c4ac-c53d-478d-be90-453b8dce9441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.415834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-78c3c4ac-c53d-478d-be90-453b8dce9441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:49.430145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1009/1995] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:49.431548 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927918d-6ab8-4f39-8183-a36b5d01c920 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.431838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927918d-6ab8-4f39-8183-a36b5d01c920 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.432385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927918d-6ab8-4f39-8183-a36b5d01c920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.452481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1927918d-6ab8-4f39-8183-a36b5d01c920 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:49.453690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1927918d-6ab8-4f39-8183-a36b5d01c920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:49.493292 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3c4ac-c53d-478d-be90-453b8dce9441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:49.494760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3c4ac-c53d-478d-be90-453b8dce9441 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:49.506035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 987/1996] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:49.506931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3c4ac-c53d-478d-be90-453b8dce9441 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:49.508656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-78c3c4ac-c53d-478d-be90-453b8dce9441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:49.549663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1010/1997] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:49.562011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927918d-6ab8-4f39-8183-a36b5d01c920 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:49.562504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:49.581328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927918d-6ab8-4f39-8183-a36b5d01c920 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:49.594738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85333643-8b5c-4d3c-971c-8620500392f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:49.595382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333643-8b5c-4d3c-971c-8620500392f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.595658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333643-8b5c-4d3c-971c-8620500392f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.595934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333643-8b5c-4d3c-971c-8620500392f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.629573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85333643-8b5c-4d3c-971c-8620500392f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:49.631276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85333643-8b5c-4d3c-971c-8620500392f5 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:49.632743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3c4ac-c53d-478d-be90-453b8dce9441 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:49.638001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502000-d9f9-429a-9353-56fc55b866aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:49.639227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502000-d9f9-429a-9353-56fc55b866aa service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.640436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502000-d9f9-429a-9353-56fc55b866aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.640840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502000-d9f9-429a-9353-56fc55b866aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.655220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f502000-d9f9-429a-9353-56fc55b866aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:49.656325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1011/1998] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:49.657573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f502000-d9f9-429a-9353-56fc55b866aa service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:49.669982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 988/1999] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:50.084704 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85333643-8b5c-4d3c-971c-8620500392f5 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:50.087570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-85333643-8b5c-4d3c-971c-8620500392f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:50.120740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85333643-8b5c-4d3c-971c-8620500392f5 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:50.141871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:50.142531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:50.142822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:50.143092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] PATH_INFO: `/v3/projects/fa4926f62db74724bbb8ba274402b10d` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:50.144641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4926f62db74724bbb8ba274402b10d)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:50.146730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:50.207353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:50.212913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502000-d9f9-429a-9353-56fc55b866aa service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:50.214698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3f502000-d9f9-429a-9353-56fc55b866aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:50.253567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502000-d9f9-429a-9353-56fc55b866aa service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:50.267664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4926f62db74724bbb8ba274402b10d', 'request_id': 'req-e2a21732-31b4-4b60-b35c-afe0b9df73e2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.275306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4926f62db74724bbb8ba274402b10d', 'request_id': 'req-e2a21732-31b4-4b60-b35c-afe0b9df73e2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.283517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4926f62db74724bbb8ba274402b10d', 'request_id': 'req-e2a21732-31b4-4b60-b35c-afe0b9df73e2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.325328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] The token cache is being invalidate because project fa4926f62db74724bbb8ba274402b10d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:50.325964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.327300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.331593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1012/2000] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:50 2022] DELETE /identity/v3/projects/fa4926f62db74724bbb8ba274402b10d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:50.334816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:50.336841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:50.417010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:50.417010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:50.417746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:50.417746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] PATH_INFO: `/v3/projects/4c63db63b5e847c29d7a0618d7a9676b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:50.421744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c63db63b5e847c29d7a0618d7a9676b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:50.424056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:50.471031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a21732-31b4-4b60-b35c-afe0b9df73e2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:50.491185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:50.493001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:50.494150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:50.494843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] PATH_INFO: `/v3/users/d0fdf6eee08f4ae48deda78cd4b52b51` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:50.497315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0fdf6eee08f4ae48deda78cd4b52b51)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:50.507547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:50.519757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c63db63b5e847c29d7a0618d7a9676b', 'request_id': 'req-5bc2ed68-058b-431d-af76-00701d9cc3ca'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.520680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c63db63b5e847c29d7a0618d7a9676b', 'request_id': 'req-5bc2ed68-058b-431d-af76-00701d9cc3ca'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.531096 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c63db63b5e847c29d7a0618d7a9676b', 'request_id': 'req-5bc2ed68-058b-431d-af76-00701d9cc3ca'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.552582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] The token cache is being invalidate because project 4c63db63b5e847c29d7a0618d7a9676b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:50.553449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.554175 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.557470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 989/2001] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:50 2022] DELETE /identity/v3/projects/4c63db63b5e847c29d7a0618d7a9676b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:50.563269 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:50.565695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:50.599465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0fdf6eee08f4ae48deda78cd4b52b51'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.600278 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0fdf6eee08f4ae48deda78cd4b52b51'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.601131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0fdf6eee08f4ae48deda78cd4b52b51'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.610459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0fdf6eee08f4ae48deda78cd4b52b51'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.621312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0fdf6eee08f4ae48deda78cd4b52b51'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.642205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1013/2002] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:50 2022] DELETE /identity/v3/users/d0fdf6eee08f4ae48deda78cd4b52b51 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:50.643134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:50.648977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:50.666869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:50.671858 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ed68-058b-431d-af76-00701d9cc3ca admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:50.712347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:50.774601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22188419-b21f-4baa-a1fe-1e3485807156 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:50.797118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ae293-4259-44e3-98e7-30c6552a0d42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:50.797755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ae293-4259-44e3-98e7-30c6552a0d42 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:50.798037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ae293-4259-44e3-98e7-30c6552a0d42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:50.798309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ae293-4259-44e3-98e7-30c6552a0d42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:50.802563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7f8ae293-4259-44e3-98e7-30c6552a0d42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:50.805507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:50.806219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:50.806507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:50.806785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] PATH_INFO: `/v3/users/9cfa653d64c547458cdeef2cadf5bbeb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:50.808287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cfa653d64c547458cdeef2cadf5bbeb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:50.810537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:50.887395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cfa653d64c547458cdeef2cadf5bbeb'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.888120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cfa653d64c547458cdeef2cadf5bbeb'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.888840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cfa653d64c547458cdeef2cadf5bbeb'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.898449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cfa653d64c547458cdeef2cadf5bbeb'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.910219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cfa653d64c547458cdeef2cadf5bbeb'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.911000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ae293-4259-44e3-98e7-30c6552a0d42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:50.914824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ae293-4259-44e3-98e7-30c6552a0d42 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:50.927582 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1014/2003] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:50.937093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 990/2004] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:50 2022] DELETE /identity/v3/users/9cfa653d64c547458cdeef2cadf5bbeb => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:51.179188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ae293-4259-44e3-98e7-30c6552a0d42 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:51.180849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7f8ae293-4259-44e3-98e7-30c6552a0d42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.213336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ae293-4259-44e3-98e7-30c6552a0d42 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:51.362962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:51.365139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.383559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d92bd9-5c97-405d-ba42-7eeb778f3cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:51.383946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d92bd9-5c97-405d-ba42-7eeb778f3cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.384308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d92bd9-5c97-405d-ba42-7eeb778f3cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.384516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d92bd9-5c97-405d-ba42-7eeb778f3cb4 admin admin] PATH_INFO: `/v3/users/75ef0e5dbd584576821298a35f1f5b05` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.385894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d92bd9-5c97-405d-ba42-7eeb778f3cb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ef0e5dbd584576821298a35f1f5b05)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:51.387803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d92bd9-5c97-405d-ba42-7eeb778f3cb4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:51.497280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aa186-b738-4589-8581-c079fdbde832 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:51.501000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-45d92bd9-5c97-405d-ba42-7eeb778f3cb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ef0e5dbd584576821298a35f1f5b05'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.501756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-45d92bd9-5c97-405d-ba42-7eeb778f3cb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ef0e5dbd584576821298a35f1f5b05'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.504296 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-45d92bd9-5c97-405d-ba42-7eeb778f3cb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ef0e5dbd584576821298a35f1f5b05'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.509490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:51.510116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.510392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.510669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] PATH_INFO: `/v3/projects/75ffe18832094f0bae945ae881e0136e` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.512430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ffe18832094f0bae945ae881e0136e)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:51.516302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:51.525791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-45d92bd9-5c97-405d-ba42-7eeb778f3cb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ef0e5dbd584576821298a35f1f5b05'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.536768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-45d92bd9-5c97-405d-ba42-7eeb778f3cb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ef0e5dbd584576821298a35f1f5b05'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.564965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1015/2005] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:51 2022] DELETE /identity/v3/users/75ef0e5dbd584576821298a35f1f5b05 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:51.568387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:51.570689 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5e8ad-ecdb-4761-abf7-c33bdc6a1094 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.571217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5e8ad-ecdb-4761-abf7-c33bdc6a1094 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.571739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5e8ad-ecdb-4761-abf7-c33bdc6a1094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.591845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:51.606837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a0b5e8ad-ecdb-4761-abf7-c33bdc6a1094 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:51.607486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ffe18832094f0bae945ae881e0136e', 'request_id': 'req-afc743bd-2446-4cad-a561-54f583235db4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.612351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a0b5e8ad-ecdb-4761-abf7-c33bdc6a1094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.618500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ffe18832094f0bae945ae881e0136e', 'request_id': 'req-afc743bd-2446-4cad-a561-54f583235db4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.633244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ffe18832094f0bae945ae881e0136e', 'request_id': 'req-afc743bd-2446-4cad-a561-54f583235db4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.663803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] The token cache is being invalidate because project 75ffe18832094f0bae945ae881e0136e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:51.664312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.666473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-afc743bd-2446-4cad-a561-54f583235db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.668430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 991/2006] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:51 2022] DELETE /identity/v3/projects/75ffe18832094f0bae945ae881e0136e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:51.681395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed032d3d-548e-4268-8dd4-410250eecb65 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.682364 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed032d3d-548e-4268-8dd4-410250eecb65 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.682650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed032d3d-548e-4268-8dd4-410250eecb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.711543 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ed032d3d-548e-4268-8dd4-410250eecb65 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:51.714408 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ed032d3d-548e-4268-8dd4-410250eecb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.792864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1016/2007] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:51.805847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5e8ad-ecdb-4761-abf7-c33bdc6a1094 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:51.807458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a0b5e8ad-ecdb-4761-abf7-c33bdc6a1094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.837087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 992/2008] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:51.848353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfe59b-bf10-49e2-8c25-1ba94b637bb7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.849036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfe59b-bf10-49e2-8c25-1ba94b637bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.849655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfe59b-bf10-49e2-8c25-1ba94b637bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.878216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5cbfe59b-bf10-49e2-8c25-1ba94b637bb7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:51.879640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5cbfe59b-bf10-49e2-8c25-1ba94b637bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.892262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 993/2009] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:51.894525 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfe59b-bf10-49e2-8c25-1ba94b637bb7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:51.895961 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5cbfe59b-bf10-49e2-8c25-1ba94b637bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.915944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfe59b-bf10-49e2-8c25-1ba94b637bb7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:51.933322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:51.934030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.934334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.934668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] PATH_INFO: `/v3/projects/2468b2485a794051a983c7973d6e8915` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.936158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2468b2485a794051a983c7973d6e8915)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:51.939132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:51.948367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5e8ad-ecdb-4761-abf7-c33bdc6a1094 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:51.953380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc032ed-9607-4de9-a9b1-e24cb9f51ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:51.954150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc032ed-9607-4de9-a9b1-e24cb9f51ddc service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.956506 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc032ed-9607-4de9-a9b1-e24cb9f51ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.956852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc032ed-9607-4de9-a9b1-e24cb9f51ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.971462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc032ed-9607-4de9-a9b1-e24cb9f51ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:51.973884 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc032ed-9607-4de9-a9b1-e24cb9f51ddc service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:51.999596 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1017/2010] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:52.000755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc032ed-9607-4de9-a9b1-e24cb9f51ddc service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.019962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2468b2485a794051a983c7973d6e8915', 'request_id': 'req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.027739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2468b2485a794051a983c7973d6e8915', 'request_id': 'req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.028583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc032ed-9607-4de9-a9b1-e24cb9f51ddc service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.034597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2468b2485a794051a983c7973d6e8915', 'request_id': 'req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.052471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] The token cache is being invalidate because project 2468b2485a794051a983c7973d6e8915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:52.052794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.054007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-64cbdaea-d5c5-4ab1-8186-bbe117c8954d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.055651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 994/2011] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:51 2022] DELETE /identity/v3/projects/2468b2485a794051a983c7973d6e8915 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:52.060676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f6a0f-a117-444b-8089-e38b1dea3cde None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.061303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f6a0f-a117-444b-8089-e38b1dea3cde None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.061816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f6a0f-a117-444b-8089-e38b1dea3cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.095549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-575f6a0f-a117-444b-8089-e38b1dea3cde None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:52.097474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-575f6a0f-a117-444b-8089-e38b1dea3cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.157819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:52.173711 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7058c-2264-4040-bf5b-493980e8ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.176547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7058c-2264-4040-bf5b-493980e8ee9f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.178687 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7058c-2264-4040-bf5b-493980e8ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.178687 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7058c-2264-4040-bf5b-493980e8ee9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.180402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7058c-2264-4040-bf5b-493980e8ee9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:52.181508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7058c-2264-4040-bf5b-493980e8ee9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.186252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7058c-2264-4040-bf5b-493980e8ee9f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.206881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1018/2012] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:52.214946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7058c-2264-4040-bf5b-493980e8ee9f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.225623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4bb7058c-2264-4040-bf5b-493980e8ee9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.227606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 995/2013] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:52.236757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea093efb-f867-4822-93dc-010a122bf098 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.237211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea093efb-f867-4822-93dc-010a122bf098 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.237595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea093efb-f867-4822-93dc-010a122bf098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.250422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7058c-2264-4040-bf5b-493980e8ee9f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.257908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69817f6-3b81-465b-9935-df1e6eca0320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.258835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69817f6-3b81-465b-9935-df1e6eca0320 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.259132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69817f6-3b81-465b-9935-df1e6eca0320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.259570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69817f6-3b81-465b-9935-df1e6eca0320 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.260654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69817f6-3b81-465b-9935-df1e6eca0320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.263885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69817f6-3b81-465b-9935-df1e6eca0320 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.274262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ea093efb-f867-4822-93dc-010a122bf098 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:52.274796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ea093efb-f867-4822-93dc-010a122bf098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.286481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 996/2014] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:52.291127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea093efb-f867-4822-93dc-010a122bf098 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.302824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1019/2015] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/projects => generated 399 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:52.307432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69817f6-3b81-465b-9935-df1e6eca0320 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.308596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea093efb-f867-4822-93dc-010a122bf098 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.347759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:52.384477 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a89a8f-1cbc-4db9-8891-777f27c26e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.385009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a89a8f-1cbc-4db9-8891-777f27c26e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.385293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a89a8f-1cbc-4db9-8891-777f27c26e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.385570 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a89a8f-1cbc-4db9-8891-777f27c26e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.386225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a89a8f-1cbc-4db9-8891-777f27c26e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:52.386533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a89a8f-1cbc-4db9-8891-777f27c26e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.388224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a89a8f-1cbc-4db9-8891-777f27c26e4c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.397488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69817f6-3b81-465b-9935-df1e6eca0320 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.402258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 997/2016] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:52.405341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c83d4-10d0-42e1-ba62-732ecebdf11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.406212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c83d4-10d0-42e1-ba62-732ecebdf11d admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.406340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a89a8f-1cbc-4db9-8891-777f27c26e4c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.406510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c83d4-10d0-42e1-ba62-732ecebdf11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.408219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c83d4-10d0-42e1-ba62-732ecebdf11d admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.409439 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c83d4-10d0-42e1-ba62-732ecebdf11d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.411517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c83d4-10d0-42e1-ba62-732ecebdf11d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.417965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a89a8f-1cbc-4db9-8891-777f27c26e4c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.423748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415f564-8f1f-4f2e-a30e-a6803939b942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.424732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415f564-8f1f-4f2e-a30e-a6803939b942 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.425021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415f564-8f1f-4f2e-a30e-a6803939b942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.425299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415f564-8f1f-4f2e-a30e-a6803939b942 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.425967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415f564-8f1f-4f2e-a30e-a6803939b942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.428155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415f564-8f1f-4f2e-a30e-a6803939b942 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.442190 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1020/2017] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:52.443309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c83d4-10d0-42e1-ba62-732ecebdf11d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.455562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c83d4-10d0-42e1-ba62-732ecebdf11d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.461764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 998/2018] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:52.461903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f919ee0-b2b0-47b6-a9d9-696b7d90d8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.462556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f919ee0-b2b0-47b6-a9d9-696b7d90d8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.462847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f919ee0-b2b0-47b6-a9d9-696b7d90d8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.463797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f919ee0-b2b0-47b6-a9d9-696b7d90d8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.463931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f919ee0-b2b0-47b6-a9d9-696b7d90d8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.465635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f919ee0-b2b0-47b6-a9d9-696b7d90d8c4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.466781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415f564-8f1f-4f2e-a30e-a6803939b942 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.481702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1021/2019] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:52.484101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f919ee0-b2b0-47b6-a9d9-696b7d90d8c4 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.494921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f919ee0-b2b0-47b6-a9d9-696b7d90d8c4 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.544814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415f564-8f1f-4f2e-a30e-a6803939b942 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.550511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd90c48-d805-4b74-a1cc-4b2d8c4783dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.551215 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd90c48-d805-4b74-a1cc-4b2d8c4783dd admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.551549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd90c48-d805-4b74-a1cc-4b2d8c4783dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.552402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd90c48-d805-4b74-a1cc-4b2d8c4783dd admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.552566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd90c48-d805-4b74-a1cc-4b2d8c4783dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.554612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd90c48-d805-4b74-a1cc-4b2d8c4783dd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.556967 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb4cc5-b7f8-49e9-8aea-0158eed890ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.557577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb4cc5-b7f8-49e9-8aea-0158eed890ba admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.557859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb4cc5-b7f8-49e9-8aea-0158eed890ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.558132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb4cc5-b7f8-49e9-8aea-0158eed890ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.558782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb4cc5-b7f8-49e9-8aea-0158eed890ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.560414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb4cc5-b7f8-49e9-8aea-0158eed890ba admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.571488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1022/2020] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:52.575576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb4cc5-b7f8-49e9-8aea-0158eed890ba admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.586965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb4cc5-b7f8-49e9-8aea-0158eed890ba admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.595740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acce6c-2211-49dd-895b-1041c4adffb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.596544 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acce6c-2211-49dd-895b-1041c4adffb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.596843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acce6c-2211-49dd-895b-1041c4adffb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.597189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acce6c-2211-49dd-895b-1041c4adffb7 admin admin] PATH_INFO: `/v3/projects/6633acea2962443aa66f3a9d1d91ade2/users/a8f723c3f8134cf69eaf43e0210b2139/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.601239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 999/2021] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/users => generated 295 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:52.605590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd90c48-d805-4b74-a1cc-4b2d8c4783dd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.619652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd90c48-d805-4b74-a1cc-4b2d8c4783dd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.626086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7a00c-fbe3-46c7-99b1-296b67ebb635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.626704 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a00c-fbe3-46c7-99b1-296b67ebb635 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.626908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acce6c-2211-49dd-895b-1041c4adffb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6633acea2962443aa66f3a9d1d91ade2, user_id=a8f723c3f8134cf69eaf43e0210b2139, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.627354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a00c-fbe3-46c7-99b1-296b67ebb635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.627354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a00c-fbe3-46c7-99b1-296b67ebb635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.627939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7a00c-fbe3-46c7-99b1-296b67ebb635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.629573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7a00c-fbe3-46c7-99b1-296b67ebb635 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.631505 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acce6c-2211-49dd-895b-1041c4adffb7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.646181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1000/2022] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:52.647082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7a00c-fbe3-46c7-99b1-296b67ebb635 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.655954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1023/2023] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:52 2022] PUT /identity/v3/projects/6633acea2962443aa66f3a9d1d91ade2/users/a8f723c3f8134cf69eaf43e0210b2139/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:52.665492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7a00c-fbe3-46c7-99b1-296b67ebb635 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.667256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d31fd-bb30-446d-a00a-59ee03522d83 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.667628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d31fd-bb30-446d-a00a-59ee03522d83 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.667934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d31fd-bb30-446d-a00a-59ee03522d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.673267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:52.707671 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-221d31fd-bb30-446d-a00a-59ee03522d83 None None] MFA Rules not processed for user `a8f723c3f8134cf69eaf43e0210b2139`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:52.709508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-221d31fd-bb30-446d-a00a-59ee03522d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.746164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dee105-eed9-4b08-8a78-dd3707eb5ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.747200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dee105-eed9-4b08-8a78-dd3707eb5ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.747713 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dee105-eed9-4b08-8a78-dd3707eb5ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.748264 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dee105-eed9-4b08-8a78-dd3707eb5ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.749644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dee105-eed9-4b08-8a78-dd3707eb5ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.752754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dee105-eed9-4b08-8a78-dd3707eb5ce0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.772410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1001/2024] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:52.774414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dee105-eed9-4b08-8a78-dd3707eb5ce0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.776862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d4dee105-eed9-4b08-8a78-dd3707eb5ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.813558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dee105-eed9-4b08-8a78-dd3707eb5ce0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.817857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1024/2025] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:52.819317 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d31fd-bb30-446d-a00a-59ee03522d83 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.826838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:52.828815 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.829893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.830413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.830759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] PATH_INFO: `/v3/projects/e2d9c20d44c643518d2e6bd565f8ccda` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.832577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d9c20d44c643518d2e6bd565f8ccda)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.835397 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.836123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d31fd-bb30-446d-a00a-59ee03522d83 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.844822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5a3f5-666d-4372-ac13-0aad541e78ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.845520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5a3f5-666d-4372-ac13-0aad541e78ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.845801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5a3f5-666d-4372-ac13-0aad541e78ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.846078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5a3f5-666d-4372-ac13-0aad541e78ea admin admin] PATH_INFO: `/v3/projects/313bccfd249a45658cf1a2343b180a37/users/321ea9ac145149f4b95dd752f896223a/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.874935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5a3f5-666d-4372-ac13-0aad541e78ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=313bccfd249a45658cf1a2343b180a37, user_id=321ea9ac145149f4b95dd752f896223a, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.877992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5a3f5-666d-4372-ac13-0aad541e78ea admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.901139 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1025/2026] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:52 2022] PUT /identity/v3/projects/313bccfd249a45658cf1a2343b180a37/users/321ea9ac145149f4b95dd752f896223a/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:52.906299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d9c20d44c643518d2e6bd565f8ccda', 'request_id': 'req-aa72c3bf-8e68-4e69-9a09-21b77a46e471'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.911696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5a3f5-666d-4372-ac13-0aad541e78ea admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.913838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e1f5a3f5-666d-4372-ac13-0aad541e78ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.917227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d9c20d44c643518d2e6bd565f8ccda', 'request_id': 'req-aa72c3bf-8e68-4e69-9a09-21b77a46e471'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.927850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d9c20d44c643518d2e6bd565f8ccda', 'request_id': 'req-aa72c3bf-8e68-4e69-9a09-21b77a46e471'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.947595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] The token cache is being invalidate because project e2d9c20d44c643518d2e6bd565f8ccda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:52.948249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.951320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.954249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1002/2027] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:52 2022] DELETE /identity/v3/projects/e2d9c20d44c643518d2e6bd565f8ccda => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:52.955363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.957150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:53.012346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5a3f5-666d-4372-ac13-0aad541e78ea admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.020182 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5910c-d64b-4b8b-8635-450485bae5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.020859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5910c-d64b-4b8b-8635-450485bae5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.021144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5910c-d64b-4b8b-8635-450485bae5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.021420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5910c-d64b-4b8b-8635-450485bae5eb admin admin] PATH_INFO: `/v3/users/aa6af53dd62248d09a8b8723df4e29f8` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.024441 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5910c-d64b-4b8b-8635-450485bae5eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa6af53dd62248d09a8b8723df4e29f8)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.027907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5910c-d64b-4b8b-8635-450485bae5eb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.051285 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72c3bf-8e68-4e69-9a09-21b77a46e471 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.059514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571eec8-2a52-481c-bdfe-939ef602cbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.060322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571eec8-2a52-481c-bdfe-939ef602cbce admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.060610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571eec8-2a52-481c-bdfe-939ef602cbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.060987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571eec8-2a52-481c-bdfe-939ef602cbce admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.062378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571eec8-2a52-481c-bdfe-939ef602cbce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.065892 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571eec8-2a52-481c-bdfe-939ef602cbce admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.095798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3dc5910c-d64b-4b8b-8635-450485bae5eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa6af53dd62248d09a8b8723df4e29f8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:53.096672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3dc5910c-d64b-4b8b-8635-450485bae5eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa6af53dd62248d09a8b8723df4e29f8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:53.097742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3dc5910c-d64b-4b8b-8635-450485bae5eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa6af53dd62248d09a8b8723df4e29f8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:53.112656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3dc5910c-d64b-4b8b-8635-450485bae5eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa6af53dd62248d09a8b8723df4e29f8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:53.128856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1003/2028] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/projects => generated 401 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:53.129022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3dc5910c-d64b-4b8b-8635-450485bae5eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa6af53dd62248d09a8b8723df4e29f8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:53.132178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5f74a-94f7-45fc-a5a6-0dfdd8e430b8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.132683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5f74a-94f7-45fc-a5a6-0dfdd8e430b8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.132683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5f74a-94f7-45fc-a5a6-0dfdd8e430b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.157393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1026/2029] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:52 2022] DELETE /identity/v3/users/aa6af53dd62248d09a8b8723df4e29f8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:53.159283 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7299706-1de3-4055-aba1-fb8a7fa04cf1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.159434 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7299706-1de3-4055-aba1-fb8a7fa04cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.159678 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7299706-1de3-4055-aba1-fb8a7fa04cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.169033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1dc5f74a-94f7-45fc-a5a6-0dfdd8e430b8 None None] MFA Rules not processed for user `321ea9ac145149f4b95dd752f896223a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:53.170300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1dc5f74a-94f7-45fc-a5a6-0dfdd8e430b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:53.179852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:53.182501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e7299706-1de3-4055-aba1-fb8a7fa04cf1 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:53.183994 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e7299706-1de3-4055-aba1-fb8a7fa04cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:53.288070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1004/2030] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:53.288999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5f74a-94f7-45fc-a5a6-0dfdd8e430b8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.292660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1dc5f74a-94f7-45fc-a5a6-0dfdd8e430b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:53.341196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:53.368876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1027/2031] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:53.373852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f8097-86dd-4cad-bce0-053f15747bd4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.374444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f8097-86dd-4cad-bce0-053f15747bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.376251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f8097-86dd-4cad-bce0-053f15747bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.420916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-dc1f8097-86dd-4cad-bce0-053f15747bd4 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:53.422544 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dc1f8097-86dd-4cad-bce0-053f15747bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:53.441865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1028/2032] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:53.445693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f8097-86dd-4cad-bce0-053f15747bd4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.447166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dc1f8097-86dd-4cad-bce0-053f15747bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:53.467654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f8097-86dd-4cad-bce0-053f15747bd4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.478118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a66ab-8de4-4f47-94cc-a3997fa23786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.479128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a66ab-8de4-4f47-94cc-a3997fa23786 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.479518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a66ab-8de4-4f47-94cc-a3997fa23786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.479952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a66ab-8de4-4f47-94cc-a3997fa23786 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.480978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a66ab-8de4-4f47-94cc-a3997fa23786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.481157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5f74a-94f7-45fc-a5a6-0dfdd8e430b8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.483307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a66ab-8de4-4f47-94cc-a3997fa23786 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.490864 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476b69e-42d4-441f-b6d4-f9b551d8b72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.491887 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476b69e-42d4-441f-b6d4-f9b551d8b72a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.491887 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476b69e-42d4-441f-b6d4-f9b551d8b72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.492216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476b69e-42d4-441f-b6d4-f9b551d8b72a admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.492994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476b69e-42d4-441f-b6d4-f9b551d8b72a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.495106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476b69e-42d4-441f-b6d4-f9b551d8b72a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.527832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1029/2033] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:53 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:53.529680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3017b-5f97-4cac-be43-b8de7abb9087 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.530177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3017b-5f97-4cac-be43-b8de7abb9087 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.530656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3017b-5f97-4cac-be43-b8de7abb9087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.537013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1005/2034] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:53 2022] POST /identity/v3/users => generated 307 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:53.540729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476b69e-42d4-441f-b6d4-f9b551d8b72a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.543116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f476b69e-42d4-441f-b6d4-f9b551d8b72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:53.560704 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d1f3017b-5f97-4cac-be43-b8de7abb9087 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:53.564512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d1f3017b-5f97-4cac-be43-b8de7abb9087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:53.649074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1030/2035] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:53.652900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3017b-5f97-4cac-be43-b8de7abb9087 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.662241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3017b-5f97-4cac-be43-b8de7abb9087 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.668950 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e283b6c-158b-4193-9f7e-a3fa2a497486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.669660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e283b6c-158b-4193-9f7e-a3fa2a497486 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.670808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e283b6c-158b-4193-9f7e-a3fa2a497486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.671221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e283b6c-158b-4193-9f7e-a3fa2a497486 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.671881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e283b6c-158b-4193-9f7e-a3fa2a497486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.674560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e283b6c-158b-4193-9f7e-a3fa2a497486 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.681348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476b69e-42d4-441f-b6d4-f9b551d8b72a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.712231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1031/2036] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:53 2022] POST /identity/v3/users => generated 300 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:53.713255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e283b6c-158b-4193-9f7e-a3fa2a497486 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.741174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e283b6c-158b-4193-9f7e-a3fa2a497486 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.751018 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f4168-eb06-4da6-a558-2de01d3fe7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.751749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f4168-eb06-4da6-a558-2de01d3fe7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.752179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f4168-eb06-4da6-a558-2de01d3fe7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.752407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f4168-eb06-4da6-a558-2de01d3fe7a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.753781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f4168-eb06-4da6-a558-2de01d3fe7a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.755401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f4168-eb06-4da6-a558-2de01d3fe7a7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.774045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1032/2037] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:53.774045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f4168-eb06-4da6-a558-2de01d3fe7a7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.790987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f4168-eb06-4da6-a558-2de01d3fe7a7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.798325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b83237-1aea-4fc8-ae44-51134897edea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.799301 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b83237-1aea-4fc8-ae44-51134897edea admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.799594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b83237-1aea-4fc8-ae44-51134897edea admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.800075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b83237-1aea-4fc8-ae44-51134897edea admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.801010 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b83237-1aea-4fc8-ae44-51134897edea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:53.802111 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b83237-1aea-4fc8-ae44-51134897edea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.804780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b83237-1aea-4fc8-ae44-51134897edea admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.805985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f0b89-1d6c-48aa-a881-44ab812a9c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.807449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0b89-1d6c-48aa-a881-44ab812a9c9b service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.807449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0b89-1d6c-48aa-a881-44ab812a9c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.807983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0b89-1d6c-48aa-a881-44ab812a9c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.825172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1033/2038] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:53.828663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f0b89-1d6c-48aa-a881-44ab812a9c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.829718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b83237-1aea-4fc8-ae44-51134897edea admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.836854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f0b89-1d6c-48aa-a881-44ab812a9c9b service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.842394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b83237-1aea-4fc8-ae44-51134897edea admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.842747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:53.850879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5d761-c3d7-48fd-8c91-7856ad235f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.851813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5d761-c3d7-48fd-8c91-7856ad235f43 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.852152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5d761-c3d7-48fd-8c91-7856ad235f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.852625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5d761-c3d7-48fd-8c91-7856ad235f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.853731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5d761-c3d7-48fd-8c91-7856ad235f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.855400 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5d761-c3d7-48fd-8c91-7856ad235f43 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.862085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1006/2039] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:53.866621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f0b89-1d6c-48aa-a881-44ab812a9c9b service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.887167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1034/2040] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:53.889194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f0b89-1d6c-48aa-a881-44ab812a9c9b service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.893204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5d761-c3d7-48fd-8c91-7856ad235f43 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.904756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a32196-b538-40ec-8f40-08c49914fcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.906410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32196-b538-40ec-8f40-08c49914fcc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.906410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32196-b538-40ec-8f40-08c49914fcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.907305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32196-b538-40ec-8f40-08c49914fcc1 admin admin] PATH_INFO: `/v3/projects/c28e9eab012244468bdfa3f98ccefb4e/users/afeaa2fd2de4453ca2ae121d62c5beab/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.928461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a32196-b538-40ec-8f40-08c49914fcc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28e9eab012244468bdfa3f98ccefb4e, user_id=afeaa2fd2de4453ca2ae121d62c5beab, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.931303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a32196-b538-40ec-8f40-08c49914fcc1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.942666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5d761-c3d7-48fd-8c91-7856ad235f43 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.952599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1007/2041] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:53 2022] PUT /identity/v3/projects/c28e9eab012244468bdfa3f98ccefb4e/users/afeaa2fd2de4453ca2ae121d62c5beab/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:53.953683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a32196-b538-40ec-8f40-08c49914fcc1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.046346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c3d19-92cf-4838-8f2a-4b0e41bc362f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.046967 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c3d19-92cf-4838-8f2a-4b0e41bc362f admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.047240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c3d19-92cf-4838-8f2a-4b0e41bc362f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.047517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c3d19-92cf-4838-8f2a-4b0e41bc362f admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.048177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c3d19-92cf-4838-8f2a-4b0e41bc362f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.051094 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c3d19-92cf-4838-8f2a-4b0e41bc362f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.102762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a32196-b538-40ec-8f40-08c49914fcc1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.107326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1035/2042] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:53 2022] POST /identity/v3/projects => generated 375 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:54.113532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c3d19-92cf-4838-8f2a-4b0e41bc362f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.118354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8ef7f-60e3-444d-9cb7-53cc5f9229c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.119051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8ef7f-60e3-444d-9cb7-53cc5f9229c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.119334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8ef7f-60e3-444d-9cb7-53cc5f9229c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.119611 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8ef7f-60e3-444d-9cb7-53cc5f9229c2 admin admin] PATH_INFO: `/v3/projects/6809380b93c14986896e8a548e48e1b2/users/49e9129bf2ba4346954a2ba8cadbed43/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.125933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:54.141717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8ef7f-60e3-444d-9cb7-53cc5f9229c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6809380b93c14986896e8a548e48e1b2, user_id=49e9129bf2ba4346954a2ba8cadbed43, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.144731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8ef7f-60e3-444d-9cb7-53cc5f9229c2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.169090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1008/2043] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:53 2022] PUT /identity/v3/projects/6809380b93c14986896e8a548e48e1b2/users/49e9129bf2ba4346954a2ba8cadbed43/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:54.170003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8ef7f-60e3-444d-9cb7-53cc5f9229c2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.181663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1de8ef7f-60e3-444d-9cb7-53cc5f9229c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.220345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c3d19-92cf-4838-8f2a-4b0e41bc362f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.278760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8ef7f-60e3-444d-9cb7-53cc5f9229c2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.285588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.286231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.286628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.286908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] PATH_INFO: `/v3/users/5617a0faa6b14b96948b9df385607ae4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.288093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5617a0faa6b14b96948b9df385607ae4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.289151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e75a12-6dc6-4c6f-80c6-b16f54085228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.289770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e75a12-6dc6-4c6f-80c6-b16f54085228 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.290052 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e75a12-6dc6-4c6f-80c6-b16f54085228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.290378 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e75a12-6dc6-4c6f-80c6-b16f54085228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.291050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e75a12-6dc6-4c6f-80c6-b16f54085228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.291390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.292681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e75a12-6dc6-4c6f-80c6-b16f54085228 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.303845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1036/2044] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:54.305854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e75a12-6dc6-4c6f-80c6-b16f54085228 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.324372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e75a12-6dc6-4c6f-80c6-b16f54085228 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.329821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd45a66-76c4-4359-b999-07a8b1418b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.330535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd45a66-76c4-4359-b999-07a8b1418b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.330850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd45a66-76c4-4359-b999-07a8b1418b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.331159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd45a66-76c4-4359-b999-07a8b1418b46 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.331841 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd45a66-76c4-4359-b999-07a8b1418b46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.333883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd45a66-76c4-4359-b999-07a8b1418b46 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.354906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5617a0faa6b14b96948b9df385607ae4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.355617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5617a0faa6b14b96948b9df385607ae4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.356646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5617a0faa6b14b96948b9df385607ae4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.367763 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1037/2045] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:54 2022] POST /identity/v3/users => generated 288 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:54.368844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5617a0faa6b14b96948b9df385607ae4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.368959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd45a66-76c4-4359-b999-07a8b1418b46 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.371556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-acd45a66-76c4-4359-b999-07a8b1418b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.375420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:54.379681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5617a0faa6b14b96948b9df385607ae4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.388868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd45a66-76c4-4359-b999-07a8b1418b46 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.395881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455df66-0241-48d9-9656-7558ee74e65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.397253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455df66-0241-48d9-9656-7558ee74e65b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.397416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455df66-0241-48d9-9656-7558ee74e65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.397838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455df66-0241-48d9-9656-7558ee74e65b admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.398603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e455df66-0241-48d9-9656-7558ee74e65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.401026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e455df66-0241-48d9-9656-7558ee74e65b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.410984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1009/2046] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:54 2022] DELETE /identity/v3/users/5617a0faa6b14b96948b9df385607ae4 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:54.412025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.413921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.417728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1038/2047] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:54.418847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455df66-0241-48d9-9656-7558ee74e65b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.421655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e455df66-0241-48d9-9656-7558ee74e65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.439063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455df66-0241-48d9-9656-7558ee74e65b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.518140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac352bd-e835-4543-bdaa-a077e296a4e0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.524029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70fe6ac-28aa-4182-84dd-948f07cd5562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.524869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fe6ac-28aa-4182-84dd-948f07cd5562 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.525275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fe6ac-28aa-4182-84dd-948f07cd5562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.525622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fe6ac-28aa-4182-84dd-948f07cd5562 admin admin] PATH_INFO: `/v3/projects/c28e9eab012244468bdfa3f98ccefb4e/users/afeaa2fd2de4453ca2ae121d62c5beab/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.527683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70fe6ac-28aa-4182-84dd-948f07cd5562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28e9eab012244468bdfa3f98ccefb4e, user_id=afeaa2fd2de4453ca2ae121d62c5beab, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.530511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70fe6ac-28aa-4182-84dd-948f07cd5562 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.543033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1010/2048] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:54 2022] PUT /identity/v3/projects/c28e9eab012244468bdfa3f98ccefb4e/users/afeaa2fd2de4453ca2ae121d62c5beab/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:54.544721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5030549-4a2b-4191-adde-eabe17c8377c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.545085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5030549-4a2b-4191-adde-eabe17c8377c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.546079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5030549-4a2b-4191-adde-eabe17c8377c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.561109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4ec5-157e-4c8f-b383-a2d31837ba0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.561799 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4ec5-157e-4c8f-b383-a2d31837ba0a admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.562096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4ec5-157e-4c8f-b383-a2d31837ba0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.562374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4ec5-157e-4c8f-b383-a2d31837ba0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.564082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4ec5-157e-4c8f-b383-a2d31837ba0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.566108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4ec5-157e-4c8f-b383-a2d31837ba0a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.574937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b5030549-4a2b-4191-adde-eabe17c8377c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:54.580325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b5030549-4a2b-4191-adde-eabe17c8377c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.583007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1039/2049] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:54.583964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4ec5-157e-4c8f-b383-a2d31837ba0a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.598621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4ec5-157e-4c8f-b383-a2d31837ba0a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.610732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:54.665795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1011/2050] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:54.667468 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5030549-4a2b-4191-adde-eabe17c8377c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.672094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b5030549-4a2b-4191-adde-eabe17c8377c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.686315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba96f6-250e-4867-bb51-5557f212fceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.687025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba96f6-250e-4867-bb51-5557f212fceb admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.687195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba96f6-250e-4867-bb51-5557f212fceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.687442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba96f6-250e-4867-bb51-5557f212fceb admin admin] PATH_INFO: `/v3/projects/6809380b93c14986896e8a548e48e1b2/users/49e9129bf2ba4346954a2ba8cadbed43/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.697993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba96f6-250e-4867-bb51-5557f212fceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6809380b93c14986896e8a548e48e1b2, user_id=49e9129bf2ba4346954a2ba8cadbed43, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.699324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5030549-4a2b-4191-adde-eabe17c8377c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.700867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba96f6-250e-4867-bb51-5557f212fceb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.723776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.723776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.723776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.723776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] PATH_INFO: `/v3/projects/963cbe680f32424f8ba6b611e4150d50` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.724291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=963cbe680f32424f8ba6b611e4150d50)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.724291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.725107 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1040/2051] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:54 2022] PUT /identity/v3/projects/6809380b93c14986896e8a548e48e1b2/users/49e9129bf2ba4346954a2ba8cadbed43/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:54.729769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2e0e3-8c70-4b05-a425-b3957d89b128 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.729769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2e0e3-8c70-4b05-a425-b3957d89b128 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.729769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2e0e3-8c70-4b05-a425-b3957d89b128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.767965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-72c2e0e3-8c70-4b05-a425-b3957d89b128 None None] MFA Rules not processed for user `afeaa2fd2de4453ca2ae121d62c5beab`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:54.769293 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-72c2e0e3-8c70-4b05-a425-b3957d89b128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.806328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963cbe680f32424f8ba6b611e4150d50', 'request_id': 'req-42db527e-dd0f-4bfb-a06b-93526cedae55'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.815833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963cbe680f32424f8ba6b611e4150d50', 'request_id': 'req-42db527e-dd0f-4bfb-a06b-93526cedae55'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.835533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963cbe680f32424f8ba6b611e4150d50', 'request_id': 'req-42db527e-dd0f-4bfb-a06b-93526cedae55'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.835533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:54.842636 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] The token cache is being invalidate because project 963cbe680f32424f8ba6b611e4150d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:54.842930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.843562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.845117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1012/2052] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:54 2022] DELETE /identity/v3/projects/963cbe680f32424f8ba6b611e4150d50 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:54.845958 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.847232 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.857465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1041/2053] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:54 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:54.858346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2e0e3-8c70-4b05-a425-b3957d89b128 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.859709 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-72c2e0e3-8c70-4b05-a425-b3957d89b128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.916692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db527e-dd0f-4bfb-a06b-93526cedae55 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.920057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2e0e3-8c70-4b05-a425-b3957d89b128 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.921578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bd136-33eb-40c1-9758-3fe74bd8a415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.922197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bd136-33eb-40c1-9758-3fe74bd8a415 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.922479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bd136-33eb-40c1-9758-3fe74bd8a415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.922817 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bd136-33eb-40c1-9758-3fe74bd8a415 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.923490 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bd136-33eb-40c1-9758-3fe74bd8a415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.925117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bd136-33eb-40c1-9758-3fe74bd8a415 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.937787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1013/2054] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:54.938949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ed640-08c8-4344-9308-a37a44c76c1a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.939237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ed640-08c8-4344-9308-a37a44c76c1a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.939517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ed640-08c8-4344-9308-a37a44c76c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.967459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:54.968561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-710ed640-08c8-4344-9308-a37a44c76c1a None None] MFA Rules not processed for user `49e9129bf2ba4346954a2ba8cadbed43`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:54.970288 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-710ed640-08c8-4344-9308-a37a44c76c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.972322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8a730-db15-4fce-badb-907490eed68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.973337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8a730-db15-4fce-badb-907490eed68c service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.973795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8a730-db15-4fce-badb-907490eed68c service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.974069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8a730-db15-4fce-badb-907490eed68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.976351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5cf8a730-db15-4fce-badb-907490eed68c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.988883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8a730-db15-4fce-badb-907490eed68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.990310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8a730-db15-4fce-badb-907490eed68c service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.000689 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1042/2055] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:55.002337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fb1aa-93ca-4728-9f0d-d1ceedfd5568 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.002337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fb1aa-93ca-4728-9f0d-d1ceedfd5568 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.002572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fb1aa-93ca-4728-9f0d-d1ceedfd5568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.021991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8b9fb1aa-93ca-4728-9f0d-d1ceedfd5568 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:55.023313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8b9fb1aa-93ca-4728-9f0d-d1ceedfd5568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.037650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1043/2056] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:55.037650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fb1aa-93ca-4728-9f0d-d1ceedfd5568 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.037650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8b9fb1aa-93ca-4728-9f0d-d1ceedfd5568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.049855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1014/2057] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:55.050419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ed640-08c8-4344-9308-a37a44c76c1a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.063375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ed640-08c8-4344-9308-a37a44c76c1a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.069936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c93f5-16b9-4e62-8273-0d27bf55969f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.071592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c93f5-16b9-4e62-8273-0d27bf55969f admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.071592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c93f5-16b9-4e62-8273-0d27bf55969f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.071874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c93f5-16b9-4e62-8273-0d27bf55969f admin admin] PATH_INFO: `/v3/projects/873a7cd008854368b11cbeda68ebd022/users/35ad868e95e94cafbbf53786b6d45b29/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.094338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c93f5-16b9-4e62-8273-0d27bf55969f admin admin] RBAC: Authorizing `identity:create_grant(project_id=873a7cd008854368b11cbeda68ebd022, user_id=35ad868e95e94cafbbf53786b6d45b29, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.097783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c93f5-16b9-4e62-8273-0d27bf55969f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.102050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fb1aa-93ca-4728-9f0d-d1ceedfd5568 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.105901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89256d-00e8-4936-983d-dd4bdff969f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.106515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89256d-00e8-4936-983d-dd4bdff969f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.107056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89256d-00e8-4936-983d-dd4bdff969f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.107322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89256d-00e8-4936-983d-dd4bdff969f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.110164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cf89256d-00e8-4936-983d-dd4bdff969f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.115106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1015/2058] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:55 2022] PUT /identity/v3/projects/873a7cd008854368b11cbeda68ebd022/users/35ad868e95e94cafbbf53786b6d45b29/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:55.116621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39075900-0e91-42ba-9531-485a9e749d62 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.117125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39075900-0e91-42ba-9531-485a9e749d62 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.117404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39075900-0e91-42ba-9531-485a9e749d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.143907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-39075900-0e91-42ba-9531-485a9e749d62 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:55.145195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-39075900-0e91-42ba-9531-485a9e749d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.188201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89256d-00e8-4936-983d-dd4bdff969f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.190318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89256d-00e8-4936-983d-dd4bdff969f7 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.201467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:55.205660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1044/2059] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:55 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:55.206492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89256d-00e8-4936-983d-dd4bdff969f7 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.208472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cf89256d-00e8-4936-983d-dd4bdff969f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.220402 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89256d-00e8-4936-983d-dd4bdff969f7 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.236851 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1016/2060] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:55.238025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad498916-e838-4813-a066-feedaec2c52f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.238291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad498916-e838-4813-a066-feedaec2c52f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.238569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad498916-e838-4813-a066-feedaec2c52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.275577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ad498916-e838-4813-a066-feedaec2c52f None None] MFA Rules not processed for user `35ad868e95e94cafbbf53786b6d45b29`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:55.277012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ad498916-e838-4813-a066-feedaec2c52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.284097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c475f-f513-449e-bbc1-f0cdcdc49e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.284854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c475f-f513-449e-bbc1-f0cdcdc49e0d service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.285203 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c475f-f513-449e-bbc1-f0cdcdc49e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.285602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c475f-f513-449e-bbc1-f0cdcdc49e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.347601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:55.369171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c475f-f513-449e-bbc1-f0cdcdc49e0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.370689 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c475f-f513-449e-bbc1-f0cdcdc49e0d service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.383590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1045/2061] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:55.385276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c475f-f513-449e-bbc1-f0cdcdc49e0d service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.387432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1017/2062] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:55.392661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad498916-e838-4813-a066-feedaec2c52f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.405474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c475f-f513-449e-bbc1-f0cdcdc49e0d service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.420696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62dfaf-f4b4-4b05-956c-3f3ab6b387a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.420696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62dfaf-f4b4-4b05-956c-3f3ab6b387a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.420696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62dfaf-f4b4-4b05-956c-3f3ab6b387a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.420696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62dfaf-f4b4-4b05-956c-3f3ab6b387a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.421256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62dfaf-f4b4-4b05-956c-3f3ab6b387a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:55.421256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62dfaf-f4b4-4b05-956c-3f3ab6b387a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.425425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62dfaf-f4b4-4b05-956c-3f3ab6b387a8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.425553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad498916-e838-4813-a066-feedaec2c52f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.431938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0dfbd-bfd9-4561-aa81-b8c2640769ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.432868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0dfbd-bfd9-4561-aa81-b8c2640769ed service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.433304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0dfbd-bfd9-4561-aa81-b8c2640769ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.433735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0dfbd-bfd9-4561-aa81-b8c2640769ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.439666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1046/2063] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:55.439666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62dfaf-f4b4-4b05-956c-3f3ab6b387a8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.440286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e6e0dfbd-bfd9-4561-aa81-b8c2640769ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.440999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6a62dfaf-f4b4-4b05-956c-3f3ab6b387a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.453705 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62dfaf-f4b4-4b05-956c-3f3ab6b387a8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.458558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.459181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.459464 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.460124 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] PATH_INFO: `/v3/projects/b5ada53236bf487f96dd65347de3ebc2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.461042 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ada53236bf487f96dd65347de3ebc2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.463047 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.510862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0dfbd-bfd9-4561-aa81-b8c2640769ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.512501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0dfbd-bfd9-4561-aa81-b8c2640769ed service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.518431 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ada53236bf487f96dd65347de3ebc2', 'request_id': 'req-795a65eb-6c0f-4507-bf70-2660491dde5d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:55.522087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1018/2064] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:55.523091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0dfbd-bfd9-4561-aa81-b8c2640769ed service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.526520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ada53236bf487f96dd65347de3ebc2', 'request_id': 'req-795a65eb-6c0f-4507-bf70-2660491dde5d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:55.534514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0dfbd-bfd9-4561-aa81-b8c2640769ed service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.540589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ada53236bf487f96dd65347de3ebc2', 'request_id': 'req-795a65eb-6c0f-4507-bf70-2660491dde5d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:55.562966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] The token cache is being invalidate because project b5ada53236bf487f96dd65347de3ebc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:55.563272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:55.567060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:55.569014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1047/2065] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:55 2022] DELETE /identity/v3/projects/b5ada53236bf487f96dd65347de3ebc2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:55.570790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.572795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.609203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaf7d5-0aec-4288-9411-48e08bcae2b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.610072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaf7d5-0aec-4288-9411-48e08bcae2b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.610349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaf7d5-0aec-4288-9411-48e08bcae2b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.610667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaf7d5-0aec-4288-9411-48e08bcae2b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.612409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93aaf7d5-0aec-4288-9411-48e08bcae2b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.661054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a65eb-6c0f-4507-bf70-2660491dde5d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.661727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:55.669282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a878f3-a969-49f4-8a92-525f5c3d76d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.669893 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a878f3-a969-49f4-8a92-525f5c3d76d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.671193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a878f3-a969-49f4-8a92-525f5c3d76d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.671478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a878f3-a969-49f4-8a92-525f5c3d76d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.672180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a878f3-a969-49f4-8a92-525f5c3d76d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.674697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a878f3-a969-49f4-8a92-525f5c3d76d0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.702315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aaf7d5-0aec-4288-9411-48e08bcae2b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.703696 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aaf7d5-0aec-4288-9411-48e08bcae2b4 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.716089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1048/2066] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:55 2022] POST /identity/v3/projects => generated 363 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:55.716089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a878f3-a969-49f4-8a92-525f5c3d76d0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.716736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1019/2067] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:55.719074 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaf7d5-0aec-4288-9411-48e08bcae2b4 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.725630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-41a878f3-a969-49f4-8a92-525f5c3d76d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.754852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaf7d5-0aec-4288-9411-48e08bcae2b4 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.821985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:55.876460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd01cd3-b25e-4e76-88ab-d33ae02800a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.877129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd01cd3-b25e-4e76-88ab-d33ae02800a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.877418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd01cd3-b25e-4e76-88ab-d33ae02800a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.877678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd01cd3-b25e-4e76-88ab-d33ae02800a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.878902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a878f3-a969-49f4-8a92-525f5c3d76d0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.879610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd01cd3-b25e-4e76-88ab-d33ae02800a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.892719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd01cd3-b25e-4e76-88ab-d33ae02800a9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.892826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.892826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.892826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.892826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] PATH_INFO: `/v3/users/b1df7fe3dcf244ada3a2b25ac86853b3` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.913364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1df7fe3dcf244ada3a2b25ac86853b3)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.915249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.934426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1020/2068] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:55 2022] POST /identity/v3/users => generated 282 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:55.939359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd01cd3-b25e-4e76-88ab-d33ae02800a9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.964526 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd01cd3-b25e-4e76-88ab-d33ae02800a9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.986071 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021db6cd-0c8b-42d4-9f26-0bd496811a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.987205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021db6cd-0c8b-42d4-9f26-0bd496811a93 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.987570 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021db6cd-0c8b-42d4-9f26-0bd496811a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.987852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021db6cd-0c8b-42d4-9f26-0bd496811a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.988614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021db6cd-0c8b-42d4-9f26-0bd496811a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.991725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021db6cd-0c8b-42d4-9f26-0bd496811a93 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.020014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1021/2069] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:56.033673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1df7fe3dcf244ada3a2b25ac86853b3'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.034466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1df7fe3dcf244ada3a2b25ac86853b3'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.035111 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1df7fe3dcf244ada3a2b25ac86853b3'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.036219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021db6cd-0c8b-42d4-9f26-0bd496811a93 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.040657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-021db6cd-0c8b-42d4-9f26-0bd496811a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.046943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1df7fe3dcf244ada3a2b25ac86853b3'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.060070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021db6cd-0c8b-42d4-9f26-0bd496811a93 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.065680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcaa68-6e0b-4218-82fe-928913372411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.066542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcaa68-6e0b-4218-82fe-928913372411 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.069941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1df7fe3dcf244ada3a2b25ac86853b3'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.074442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcaa68-6e0b-4218-82fe-928913372411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.078255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcaa68-6e0b-4218-82fe-928913372411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.079090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcaa68-6e0b-4218-82fe-928913372411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.081725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcaa68-6e0b-4218-82fe-928913372411 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.108899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1049/2070] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:55 2022] DELETE /identity/v3/users/b1df7fe3dcf244ada3a2b25ac86853b3 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:56.114121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1022/2071] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:56.115766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.172299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcaa68-6e0b-4218-82fe-928913372411 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.174858 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-01dcaa68-6e0b-4218-82fe-928913372411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.199621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcaa68-6e0b-4218-82fe-928913372411 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.243044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8859262-6c4a-42d8-b775-66b27797a76e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.249600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14073d-5a28-4d4f-90ce-548420667875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.250424 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14073d-5a28-4d4f-90ce-548420667875 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.250799 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14073d-5a28-4d4f-90ce-548420667875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.251198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14073d-5a28-4d4f-90ce-548420667875 admin admin] PATH_INFO: `/v3/projects/7c003a4052364861a10ac993a3e58ad2/users/a99c0bc8fd2443dbaa41d47b7e5c22ab/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.271665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14073d-5a28-4d4f-90ce-548420667875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c003a4052364861a10ac993a3e58ad2, user_id=a99c0bc8fd2443dbaa41d47b7e5c22ab, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.275482 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14073d-5a28-4d4f-90ce-548420667875 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.308820 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1050/2072] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:56 2022] PUT /identity/v3/projects/7c003a4052364861a10ac993a3e58ad2/users/a99c0bc8fd2443dbaa41d47b7e5c22ab/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:56.321833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b7318-3b17-4022-ae56-dee0011ca961 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.322588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b7318-3b17-4022-ae56-dee0011ca961 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.322882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b7318-3b17-4022-ae56-dee0011ca961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.328947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb96ed4-7057-4a71-b8ba-5fc8a38aeeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.329650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb96ed4-7057-4a71-b8ba-5fc8a38aeeca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.329987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb96ed4-7057-4a71-b8ba-5fc8a38aeeca admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.330335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb96ed4-7057-4a71-b8ba-5fc8a38aeeca admin admin] PATH_INFO: `/v3/users/321ea9ac145149f4b95dd752f896223a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.332901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb96ed4-7057-4a71-b8ba-5fc8a38aeeca admin admin] RBAC: Authorizing `identity:delete_user(user_id=321ea9ac145149f4b95dd752f896223a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.335107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb96ed4-7057-4a71-b8ba-5fc8a38aeeca admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.356716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d20b7318-3b17-4022-ae56-dee0011ca961 None None] MFA Rules not processed for user `a99c0bc8fd2443dbaa41d47b7e5c22ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:56.358147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d20b7318-3b17-4022-ae56-dee0011ca961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.388097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:56.391855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7cb96ed4-7057-4a71-b8ba-5fc8a38aeeca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '321ea9ac145149f4b95dd752f896223a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.393864 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7cb96ed4-7057-4a71-b8ba-5fc8a38aeeca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '321ea9ac145149f4b95dd752f896223a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.394551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7cb96ed4-7057-4a71-b8ba-5fc8a38aeeca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '321ea9ac145149f4b95dd752f896223a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.399954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:56.404451 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7cb96ed4-7057-4a71-b8ba-5fc8a38aeeca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '321ea9ac145149f4b95dd752f896223a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.414378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7cb96ed4-7057-4a71-b8ba-5fc8a38aeeca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '321ea9ac145149f4b95dd752f896223a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.434399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1023/2073] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:56 2022] DELETE /identity/v3/users/321ea9ac145149f4b95dd752f896223a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:56.439658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937908a-1c7d-41df-b2e0-291a49da71f1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.440116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937908a-1c7d-41df-b2e0-291a49da71f1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.440261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937908a-1c7d-41df-b2e0-291a49da71f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.450264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1051/2074] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:56.463394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b7318-3b17-4022-ae56-dee0011ca961 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.465156 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d20b7318-3b17-4022-ae56-dee0011ca961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.467374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6937908a-1c7d-41df-b2e0-291a49da71f1 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:56.468988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6937908a-1c7d-41df-b2e0-291a49da71f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.535820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1024/2075] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:56.536714 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937908a-1c7d-41df-b2e0-291a49da71f1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.540349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6937908a-1c7d-41df-b2e0-291a49da71f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.542632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b7318-3b17-4022-ae56-dee0011ca961 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.547315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.547943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.548260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.548534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] PATH_INFO: `/v3/projects/067a44dec03945cfab82b08094dfbf46` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.549648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=067a44dec03945cfab82b08094dfbf46)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.551550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.603705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937908a-1c7d-41df-b2e0-291a49da71f1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.608081 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '067a44dec03945cfab82b08094dfbf46', 'request_id': 'req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.615839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '067a44dec03945cfab82b08094dfbf46', 'request_id': 'req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.621858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '067a44dec03945cfab82b08094dfbf46', 'request_id': 'req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.635547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] The token cache is being invalidate because project 067a44dec03945cfab82b08094dfbf46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:56.635837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.636524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.640832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1052/2076] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:56 2022] DELETE /identity/v3/projects/067a44dec03945cfab82b08094dfbf46 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:56.640832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.640832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.655848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ff665-6823-4d81-9822-26f1634af771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.656486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ff665-6823-4d81-9822-26f1634af771 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.656765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ff665-6823-4d81-9822-26f1634af771 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.657035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ff665-6823-4d81-9822-26f1634af771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.659257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ad7ff665-6823-4d81-9822-26f1634af771 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.693864 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:56.698217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf42cf9-42bd-4b45-93ea-80a0a60f70f9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.702224 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335924b-a2e8-4496-b3e3-a318d41b73bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.702825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335924b-a2e8-4496-b3e3-a318d41b73bd service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.703103 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335924b-a2e8-4496-b3e3-a318d41b73bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.703376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335924b-a2e8-4496-b3e3-a318d41b73bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.705198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8335924b-a2e8-4496-b3e3-a318d41b73bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.719246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:56.725169 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ff665-6823-4d81-9822-26f1634af771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.726627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ff665-6823-4d81-9822-26f1634af771 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.736909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1025/2077] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:56.737901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ff665-6823-4d81-9822-26f1634af771 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.739344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ad7ff665-6823-4d81-9822-26f1634af771 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.792365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335924b-a2e8-4496-b3e3-a318d41b73bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.793860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335924b-a2e8-4496-b3e3-a318d41b73bd service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.805793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1053/2078] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:56.820204 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ff665-6823-4d81-9822-26f1634af771 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.825782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.827205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.827500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.827801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] PATH_INFO: `/v3/users/d92424f4248e4d06b8755bb4d074e69c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.828995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d92424f4248e4d06b8755bb4d074e69c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.830916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.882901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd92424f4248e4d06b8755bb4d074e69c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.883777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd92424f4248e4d06b8755bb4d074e69c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.884453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd92424f4248e4d06b8755bb4d074e69c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.895761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd92424f4248e4d06b8755bb4d074e69c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.907918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd92424f4248e4d06b8755bb4d074e69c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.933762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1026/2079] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:56 2022] DELETE /identity/v3/users/d92424f4248e4d06b8755bb4d074e69c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:57.290167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335924b-a2e8-4496-b3e3-a318d41b73bd service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:57.291616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8335924b-a2e8-4496-b3e3-a318d41b73bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.311643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335924b-a2e8-4496-b3e3-a318d41b73bd service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:57.354793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:57.357010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.444351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:57.444818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.445816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.446013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] PATH_INFO: `/v3/projects/313bccfd249a45658cf1a2343b180a37` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.448762 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=313bccfd249a45658cf1a2343b180a37)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:57.450983 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:57.465668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:57.495745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1b8dc-cfe5-49bc-8b35-9115ea6cbbd7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:57.507654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:57.508730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.509170 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.509744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] PATH_INFO: `/v3/projects/58a3f2c2e4164917a6413f44dfe7c006` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.510927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a3f2c2e4164917a6413f44dfe7c006)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:57.512898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:57.529171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '313bccfd249a45658cf1a2343b180a37', 'request_id': 'req-6c9214a3-58b6-41e9-944c-d91e45d09d18'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.541318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '313bccfd249a45658cf1a2343b180a37', 'request_id': 'req-6c9214a3-58b6-41e9-944c-d91e45d09d18'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.558078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '313bccfd249a45658cf1a2343b180a37', 'request_id': 'req-6c9214a3-58b6-41e9-944c-d91e45d09d18'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.580442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] The token cache is being invalidate because project 313bccfd249a45658cf1a2343b180a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:57.580930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.582044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.589503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1054/2080] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:57 2022] DELETE /identity/v3/projects/313bccfd249a45658cf1a2343b180a37 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:57.592361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:57.595455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.625825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a3f2c2e4164917a6413f44dfe7c006', 'request_id': 'req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.639984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a3f2c2e4164917a6413f44dfe7c006', 'request_id': 'req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.646614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a3f2c2e4164917a6413f44dfe7c006', 'request_id': 'req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.666031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] The token cache is being invalidate because project 58a3f2c2e4164917a6413f44dfe7c006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:57.666656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.667688 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f4e78f7-c6c0-4c6f-bf4e-55ff805eead5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.669628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1027/2081] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:57 2022] DELETE /identity/v3/projects/58a3f2c2e4164917a6413f44dfe7c006 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:57.678899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b2ab2-e025-4a95-adb6-dd2ae9949c57 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.679239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b2ab2-e025-4a95-adb6-dd2ae9949c57 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.679543 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b2ab2-e025-4a95-adb6-dd2ae9949c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.693193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:57.707672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b38b2ab2-e025-4a95-adb6-dd2ae9949c57 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:57.708924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b38b2ab2-e025-4a95-adb6-dd2ae9949c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.726159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9214a3-58b6-41e9-944c-d91e45d09d18 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:57.797992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1028/2082] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:57.801941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488758c6-32dc-4f2e-991b-d7b35488f901 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.802234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488758c6-32dc-4f2e-991b-d7b35488f901 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.802532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488758c6-32dc-4f2e-991b-d7b35488f901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.808392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:57.809033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.809317 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.809593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] PATH_INFO: `/v3/users/49e9129bf2ba4346954a2ba8cadbed43` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.810918 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e9129bf2ba4346954a2ba8cadbed43)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:57.813228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:57.822796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-488758c6-32dc-4f2e-991b-d7b35488f901 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:57.824033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-488758c6-32dc-4f2e-991b-d7b35488f901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.833598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1029/2083] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:57.837592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488758c6-32dc-4f2e-991b-d7b35488f901 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:57.851055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488758c6-32dc-4f2e-991b-d7b35488f901 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:57.856629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47dd06-8dd3-4463-bc18-4f8c36d814d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:57.857243 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47dd06-8dd3-4463-bc18-4f8c36d814d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.858476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47dd06-8dd3-4463-bc18-4f8c36d814d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.858771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47dd06-8dd3-4463-bc18-4f8c36d814d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.859421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47dd06-8dd3-4463-bc18-4f8c36d814d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:57.859994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47dd06-8dd3-4463-bc18-4f8c36d814d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:57.861717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47dd06-8dd3-4463-bc18-4f8c36d814d9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:57.871329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e9129bf2ba4346954a2ba8cadbed43'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.872088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e9129bf2ba4346954a2ba8cadbed43'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.873287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e9129bf2ba4346954a2ba8cadbed43'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.878851 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1030/2084] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:03:57.884895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e9129bf2ba4346954a2ba8cadbed43'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.885404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47dd06-8dd3-4463-bc18-4f8c36d814d9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:57.887293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-be47dd06-8dd3-4463-bc18-4f8c36d814d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.892951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e9129bf2ba4346954a2ba8cadbed43'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.905128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47dd06-8dd3-4463-bc18-4f8c36d814d9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:57.913040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1055/2085] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:57 2022] DELETE /identity/v3/users/49e9129bf2ba4346954a2ba8cadbed43 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:57.917937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960f35f-54fb-4325-bb65-f76d11d0fc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:57.918552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960f35f-54fb-4325-bb65-f76d11d0fc70 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.918835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960f35f-54fb-4325-bb65-f76d11d0fc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.919110 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960f35f-54fb-4325-bb65-f76d11d0fc70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.919747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960f35f-54fb-4325-bb65-f76d11d0fc70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:57.922327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960f35f-54fb-4325-bb65-f76d11d0fc70 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:57.961825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1031/2086] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:57 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:57.965689 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:58.081234 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9d259-c2d8-4a92-a20b-b0e8f0433335 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:58.100332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:58.209726 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7957a-98fd-4698-a863-5518373d157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:58.209726 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7957a-98fd-4698-a863-5518373d157c admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.209726 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7957a-98fd-4698-a863-5518373d157c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.209726 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7957a-98fd-4698-a863-5518373d157c admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.210299 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7957a-98fd-4698-a863-5518373d157c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:58.227046 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7957a-98fd-4698-a863-5518373d157c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:58.281731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1056/2087] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:57 2022] POST /identity/v3/users => generated 290 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:58.289817 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960f35f-54fb-4325-bb65-f76d11d0fc70 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:58.302524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7957a-98fd-4698-a863-5518373d157c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:58.305123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960f35f-54fb-4325-bb65-f76d11d0fc70 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:58.311758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a4b7957a-98fd-4698-a863-5518373d157c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:58.322629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2490cd-048a-44d9-86e6-4c1a48322b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:58.323226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2490cd-048a-44d9-86e6-4c1a48322b99 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.323514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2490cd-048a-44d9-86e6-4c1a48322b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.324594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2490cd-048a-44d9-86e6-4c1a48322b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.325227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2490cd-048a-44d9-86e6-4c1a48322b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:58.327523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2490cd-048a-44d9-86e6-4c1a48322b99 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:58.341095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7957a-98fd-4698-a863-5518373d157c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:58.354231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:58.361754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.362076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.362360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] PATH_INFO: `/v3/projects/6809380b93c14986896e8a548e48e1b2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.364084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6809380b93c14986896e8a548e48e1b2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:58.364199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1032/2088] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:58.365156 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2490cd-048a-44d9-86e6-4c1a48322b99 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:58.368310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:58.379516 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2490cd-048a-44d9-86e6-4c1a48322b99 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:58.392539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafd19f-a627-4585-a9f3-42dc8ecfb77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:58.393695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafd19f-a627-4585-a9f3-42dc8ecfb77f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.394249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafd19f-a627-4585-a9f3-42dc8ecfb77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.394736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafd19f-a627-4585-a9f3-42dc8ecfb77f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.395753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baafd19f-a627-4585-a9f3-42dc8ecfb77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:58.397851 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baafd19f-a627-4585-a9f3-42dc8ecfb77f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:58.411380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1033/2089] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:58.415921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafd19f-a627-4585-a9f3-42dc8ecfb77f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:58.438819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafd19f-a627-4585-a9f3-42dc8ecfb77f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:58.447469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6809380b93c14986896e8a548e48e1b2', 'request_id': 'req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.457696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6809380b93c14986896e8a548e48e1b2', 'request_id': 'req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.463998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6809380b93c14986896e8a548e48e1b2', 'request_id': 'req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.478421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:58.487502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] The token cache is being invalidate because project 6809380b93c14986896e8a548e48e1b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:58.487931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.488824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6931c6d6-11db-4cee-b6ba-ce77c37d9c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.495936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1057/2090] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:58 2022] DELETE /identity/v3/projects/6809380b93c14986896e8a548e48e1b2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:58.501617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d985f8-341f-4f9e-bd95-cab3a9cf9842 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.502076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d985f8-341f-4f9e-bd95-cab3a9cf9842 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.502508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d985f8-341f-4f9e-bd95-cab3a9cf9842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.540073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f4d985f8-341f-4f9e-bd95-cab3a9cf9842 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:58.541518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f4d985f8-341f-4f9e-bd95-cab3a9cf9842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:58.566567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f091d-bef7-4500-b505-b81d5e9add52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:58.567244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f091d-bef7-4500-b505-b81d5e9add52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.567526 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f091d-bef7-4500-b505-b81d5e9add52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.567806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f091d-bef7-4500-b505-b81d5e9add52 admin admin] PATH_INFO: `/v3/projects/b7f27993d4264fbeaba7d1888ada9599/users/8e04acfc2e4340f8afcc8ce8ffb4c983/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.585707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f091d-bef7-4500-b505-b81d5e9add52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f27993d4264fbeaba7d1888ada9599, user_id=8e04acfc2e4340f8afcc8ce8ffb4c983, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:58.592378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f091d-bef7-4500-b505-b81d5e9add52 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:58.616261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1034/2091] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:58 2022] PUT /identity/v3/projects/b7f27993d4264fbeaba7d1888ada9599/users/8e04acfc2e4340f8afcc8ce8ffb4c983/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:58.628448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0e54-4689-4b33-95de-cc5fb1673755 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.628739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0e54-4689-4b33-95de-cc5fb1673755 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.629018 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0e54-4689-4b33-95de-cc5fb1673755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.665438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c80b0e54-4689-4b33-95de-cc5fb1673755 None None] MFA Rules not processed for user `8e04acfc2e4340f8afcc8ce8ffb4c983`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:58.668770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c80b0e54-4689-4b33-95de-cc5fb1673755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:58.678475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1058/2092] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:58.686896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3abd2c-53e0-48ef-8ba8-1fcca68637df None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.687189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3abd2c-53e0-48ef-8ba8-1fcca68637df None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.687465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3abd2c-53e0-48ef-8ba8-1fcca68637df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.715608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-9f3abd2c-53e0-48ef-8ba8-1fcca68637df None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:58.730325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9f3abd2c-53e0-48ef-8ba8-1fcca68637df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:58.769952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:58.830150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1035/2093] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:58.834771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b0e54-4689-4b33-95de-cc5fb1673755 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:58.836407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c80b0e54-4689-4b33-95de-cc5fb1673755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:58.863201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1059/2094] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:58.867613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3abd2c-53e0-48ef-8ba8-1fcca68637df None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:58.885580 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3abd2c-53e0-48ef-8ba8-1fcca68637df None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:58.900178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b911142-6caa-48e8-af67-666a56869fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:58.902266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b911142-6caa-48e8-af67-666a56869fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.903032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b911142-6caa-48e8-af67-666a56869fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.903508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b911142-6caa-48e8-af67-666a56869fd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.905193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b911142-6caa-48e8-af67-666a56869fd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:58.905771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b911142-6caa-48e8-af67-666a56869fd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:58.908121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b911142-6caa-48e8-af67-666a56869fd2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:58.927249 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1060/2095] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:58.931804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b911142-6caa-48e8-af67-666a56869fd2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:58.951448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b0e54-4689-4b33-95de-cc5fb1673755 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:58.952294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b911142-6caa-48e8-af67-666a56869fd2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:58.958217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e49dbd-44c9-49d0-8bb8-e617f350ca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:58.958923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e49dbd-44c9-49d0-8bb8-e617f350ca15 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.959319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e49dbd-44c9-49d0-8bb8-e617f350ca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.959643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e49dbd-44c9-49d0-8bb8-e617f350ca15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.960420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e49dbd-44c9-49d0-8bb8-e617f350ca15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:58.962536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e49dbd-44c9-49d0-8bb8-e617f350ca15 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:58.963912 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ede34-7961-4879-b976-2ce4c639e130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:58.965386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ede34-7961-4879-b976-2ce4c639e130 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.965935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ede34-7961-4879-b976-2ce4c639e130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.967236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ede34-7961-4879-b976-2ce4c639e130 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.968440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ede34-7961-4879-b976-2ce4c639e130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:58.971492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ede34-7961-4879-b976-2ce4c639e130 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.023062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1036/2096] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:58 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:59.026471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1061/2097] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:58 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:59.030579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e49dbd-44c9-49d0-8bb8-e617f350ca15 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.037322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ede34-7961-4879-b976-2ce4c639e130 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.068737 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:59.110672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e49dbd-44c9-49d0-8bb8-e617f350ca15 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.114367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ede34-7961-4879-b976-2ce4c639e130 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.115327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11175da2-07d5-4e8e-a2e9-6869c34b7b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.115926 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11175da2-07d5-4e8e-a2e9-6869c34b7b97 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.116245 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11175da2-07d5-4e8e-a2e9-6869c34b7b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.116712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11175da2-07d5-4e8e-a2e9-6869c34b7b97 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.117361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11175da2-07d5-4e8e-a2e9-6869c34b7b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.119290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11175da2-07d5-4e8e-a2e9-6869c34b7b97 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.119660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef357e1-a363-49f2-b908-ddedceeec1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.120293 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef357e1-a363-49f2-b908-ddedceeec1f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.120573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef357e1-a363-49f2-b908-ddedceeec1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.120844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef357e1-a363-49f2-b908-ddedceeec1f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.121473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef357e1-a363-49f2-b908-ddedceeec1f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.124140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef357e1-a363-49f2-b908-ddedceeec1f1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.146625 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1037/2098] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:59.147738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1062/2099] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:59.150833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11175da2-07d5-4e8e-a2e9-6869c34b7b97 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.154272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef357e1-a363-49f2-b908-ddedceeec1f1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.161079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11175da2-07d5-4e8e-a2e9-6869c34b7b97 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.164877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef357e1-a363-49f2-b908-ddedceeec1f1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.166745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682b6b7-d76c-4bbf-b680-47457fc26909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.167347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682b6b7-d76c-4bbf-b680-47457fc26909 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.167831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682b6b7-d76c-4bbf-b680-47457fc26909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.168135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682b6b7-d76c-4bbf-b680-47457fc26909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.168788 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682b6b7-d76c-4bbf-b680-47457fc26909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.170304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3a1da-e571-4193-b825-519fe81b3061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.170888 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682b6b7-d76c-4bbf-b680-47457fc26909 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.170995 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3a1da-e571-4193-b825-519fe81b3061 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.171189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3a1da-e571-4193-b825-519fe81b3061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.171461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3a1da-e571-4193-b825-519fe81b3061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.172344 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3a1da-e571-4193-b825-519fe81b3061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.173776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3a1da-e571-4193-b825-519fe81b3061 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.186729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1063/2100] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:59.188830 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1038/2101] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:59.191459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3a1da-e571-4193-b825-519fe81b3061 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.192525 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682b6b7-d76c-4bbf-b680-47457fc26909 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.202021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3a1da-e571-4193-b825-519fe81b3061 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.202362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682b6b7-d76c-4bbf-b680-47457fc26909 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.207138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd5423-71f2-46ed-b40b-7ba9ca2baf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.207292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce3f2b-9238-47e4-bfe9-eb2248453564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.207770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd5423-71f2-46ed-b40b-7ba9ca2baf88 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.207969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce3f2b-9238-47e4-bfe9-eb2248453564 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.208088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce3f2b-9238-47e4-bfe9-eb2248453564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.208167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd5423-71f2-46ed-b40b-7ba9ca2baf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.208352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce3f2b-9238-47e4-bfe9-eb2248453564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.208465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd5423-71f2-46ed-b40b-7ba9ca2baf88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.209010 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce3f2b-9238-47e4-bfe9-eb2248453564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.209116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd5423-71f2-46ed-b40b-7ba9ca2baf88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.210769 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd5423-71f2-46ed-b40b-7ba9ca2baf88 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.211108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce3f2b-9238-47e4-bfe9-eb2248453564 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.223295 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1039/2102] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:59.225112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1064/2103] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:59.227389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd5423-71f2-46ed-b40b-7ba9ca2baf88 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.228960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce3f2b-9238-47e4-bfe9-eb2248453564 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.239566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd5423-71f2-46ed-b40b-7ba9ca2baf88 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.240573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce3f2b-9238-47e4-bfe9-eb2248453564 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.247237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c9391-7044-4a62-add1-c087ff5b9b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.247977 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c9391-7044-4a62-add1-c087ff5b9b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.248286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c9391-7044-4a62-add1-c087ff5b9b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.248481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b82a84-0279-4d0e-b6e7-d8b940334164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.248598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c9391-7044-4a62-add1-c087ff5b9b26 admin admin] PATH_INFO: `/v3/projects/5f31b79049464a7193d85f1dc03b9f0a/users/5660fc040b204d66ba1b2e9ce52dfb6e/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.249137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b82a84-0279-4d0e-b6e7-d8b940334164 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.249499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b82a84-0279-4d0e-b6e7-d8b940334164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.249844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b82a84-0279-4d0e-b6e7-d8b940334164 admin admin] PATH_INFO: `/v3/projects/aba80db304d146ab94932249d1515e72/users/eaae66d2577b43ebaf9d3cf257559bf7/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.266553 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b82a84-0279-4d0e-b6e7-d8b940334164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba80db304d146ab94932249d1515e72, user_id=eaae66d2577b43ebaf9d3cf257559bf7, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.269485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b82a84-0279-4d0e-b6e7-d8b940334164 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.269612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c9391-7044-4a62-add1-c087ff5b9b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f31b79049464a7193d85f1dc03b9f0a, user_id=5660fc040b204d66ba1b2e9ce52dfb6e, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.272535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c9391-7044-4a62-add1-c087ff5b9b26 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.286276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1065/2104] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:59 2022] PUT /identity/v3/projects/aba80db304d146ab94932249d1515e72/users/eaae66d2577b43ebaf9d3cf257559bf7/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:59.289458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1040/2105] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:59 2022] PUT /identity/v3/projects/5f31b79049464a7193d85f1dc03b9f0a/users/5660fc040b204d66ba1b2e9ce52dfb6e/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:59.300572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39043308-e7e6-4a01-a87c-0eaf63749d0c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.300572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39043308-e7e6-4a01-a87c-0eaf63749d0c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.300572 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39043308-e7e6-4a01-a87c-0eaf63749d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.302855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c9391-7044-4a62-add1-c087ff5b9b26 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.304639 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7f7c9391-7044-4a62-add1-c087ff5b9b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.334812 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-39043308-e7e6-4a01-a87c-0eaf63749d0c None None] MFA Rules not processed for user `eaae66d2577b43ebaf9d3cf257559bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:59.336037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-39043308-e7e6-4a01-a87c-0eaf63749d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.375364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:59.398988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c9391-7044-4a62-add1-c087ff5b9b26 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.403902 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.405076 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.405354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.405620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] PATH_INFO: `/v3/users/35ad868e95e94cafbbf53786b6d45b29` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.406770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35ad868e95e94cafbbf53786b6d45b29)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.408732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.423564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1066/2106] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:59.428190 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232b116-08cc-4aa3-97cd-ce8ab9fcd0ba None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.428473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232b116-08cc-4aa3-97cd-ce8ab9fcd0ba None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.428746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232b116-08cc-4aa3-97cd-ce8ab9fcd0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.456886 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35ad868e95e94cafbbf53786b6d45b29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.457724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35ad868e95e94cafbbf53786b6d45b29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.458351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35ad868e95e94cafbbf53786b6d45b29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.458636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7232b116-08cc-4aa3-97cd-ce8ab9fcd0ba None None] MFA Rules not processed for user `5660fc040b204d66ba1b2e9ce52dfb6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:59.459847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7232b116-08cc-4aa3-97cd-ce8ab9fcd0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.466208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35ad868e95e94cafbbf53786b6d45b29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.472304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:59.474583 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35ad868e95e94cafbbf53786b6d45b29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.495638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1041/2107] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:59 2022] DELETE /identity/v3/users/35ad868e95e94cafbbf53786b6d45b29 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:59.498650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.498650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.532357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1067/2108] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:59.533600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f5b70-043f-4779-8f62-3329ba4e02b9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.533889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f5b70-043f-4779-8f62-3329ba4e02b9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.534163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f5b70-043f-4779-8f62-3329ba4e02b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.552580 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-316f5b70-043f-4779-8f62-3329ba4e02b9 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:59.553797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-316f5b70-043f-4779-8f62-3329ba4e02b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.568277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99383f6-9f13-4fb0-8c52-ab5bd8123284 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.572915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719f20a-96c2-4595-9cae-b529b287e1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.573516 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719f20a-96c2-4595-9cae-b529b287e1d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.573795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719f20a-96c2-4595-9cae-b529b287e1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.574069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719f20a-96c2-4595-9cae-b529b287e1d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.574692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719f20a-96c2-4595-9cae-b529b287e1d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.576837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719f20a-96c2-4595-9cae-b529b287e1d8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.605280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1042/2109] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:59.606353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719f20a-96c2-4595-9cae-b529b287e1d8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.608524 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9719f20a-96c2-4595-9cae-b529b287e1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.617731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1068/2110] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:59.618692 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f5b70-043f-4779-8f62-3329ba4e02b9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.628935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f5b70-043f-4779-8f62-3329ba4e02b9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.682498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:59.682720 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719f20a-96c2-4595-9cae-b529b287e1d8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.687738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd495d4-8963-4821-b356-fc758b7001cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.688364 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd495d4-8963-4821-b356-fc758b7001cd admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.688642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd495d4-8963-4821-b356-fc758b7001cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.688912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd495d4-8963-4821-b356-fc758b7001cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.689407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63141704-2af6-4197-aeb9-8674aaa1a0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.689522 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd495d4-8963-4821-b356-fc758b7001cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.690001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63141704-2af6-4197-aeb9-8674aaa1a0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.690280 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63141704-2af6-4197-aeb9-8674aaa1a0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.690549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63141704-2af6-4197-aeb9-8674aaa1a0e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.691204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63141704-2af6-4197-aeb9-8674aaa1a0e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.691416 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd495d4-8963-4821-b356-fc758b7001cd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.693320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63141704-2af6-4197-aeb9-8674aaa1a0e3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.725689 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1069/2111] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:59.726698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63141704-2af6-4197-aeb9-8674aaa1a0e3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.729813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-63141704-2af6-4197-aeb9-8674aaa1a0e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.733517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1043/2112] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:59.733836 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd495d4-8963-4821-b356-fc758b7001cd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.803335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:59.804298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63141704-2af6-4197-aeb9-8674aaa1a0e3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.832374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd495d4-8963-4821-b356-fc758b7001cd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.840928 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f42ec-1331-4947-bf11-22a8a671b7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.843040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f42ec-1331-4947-bf11-22a8a671b7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.843326 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f42ec-1331-4947-bf11-22a8a671b7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.843604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f42ec-1331-4947-bf11-22a8a671b7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.844313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f42ec-1331-4947-bf11-22a8a671b7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.847034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f42ec-1331-4947-bf11-22a8a671b7e6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.863757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1044/2113] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:59.863757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f42ec-1331-4947-bf11-22a8a671b7e6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.872308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2734af1-5d21-420a-b8f1-845dea2e5647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.873213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2734af1-5d21-420a-b8f1-845dea2e5647 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.873642 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2734af1-5d21-420a-b8f1-845dea2e5647 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.874209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2734af1-5d21-420a-b8f1-845dea2e5647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.879733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f42ec-1331-4947-bf11-22a8a671b7e6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.896275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2734af1-5d21-420a-b8f1-845dea2e5647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.896275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2734af1-5d21-420a-b8f1-845dea2e5647 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.896474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795a632-69ab-4f75-8366-047bff4d912b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.896474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795a632-69ab-4f75-8366-047bff4d912b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.896474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795a632-69ab-4f75-8366-047bff4d912b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.896474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795a632-69ab-4f75-8366-047bff4d912b admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.896817 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795a632-69ab-4f75-8366-047bff4d912b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.896817 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795a632-69ab-4f75-8366-047bff4d912b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.900383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1070/2114] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:59.907881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2734af1-5d21-420a-b8f1-845dea2e5647 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.920805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2734af1-5d21-420a-b8f1-845dea2e5647 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.925835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fec96-36d5-438c-8e80-8df6cb5a71d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.926650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fec96-36d5-438c-8e80-8df6cb5a71d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.926939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fec96-36d5-438c-8e80-8df6cb5a71d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.927218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fec96-36d5-438c-8e80-8df6cb5a71d9 admin admin] PATH_INFO: `/v3/projects/22730d1bea7d4b919643a9acfc534ffe/users/f340d1e83b4545dba8dcef466afb6fd5/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.929856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1045/2115] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/users => generated 300 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:59.934313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795a632-69ab-4f75-8366-047bff4d912b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.958923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3fec96-36d5-438c-8e80-8df6cb5a71d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22730d1bea7d4b919643a9acfc534ffe, user_id=f340d1e83b4545dba8dcef466afb6fd5, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.963027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795a632-69ab-4f75-8366-047bff4d912b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.963248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3fec96-36d5-438c-8e80-8df6cb5a71d9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.972299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4178aa8-58af-4be2-bba5-141070f26035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.973475 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4178aa8-58af-4be2-bba5-141070f26035 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.974034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4178aa8-58af-4be2-bba5-141070f26035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.975400 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4178aa8-58af-4be2-bba5-141070f26035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.976505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4178aa8-58af-4be2-bba5-141070f26035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.987889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1071/2116] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:59 2022] PUT /identity/v3/projects/22730d1bea7d4b919643a9acfc534ffe/users/f340d1e83b4545dba8dcef466afb6fd5/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:59.991812 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fec96-36d5-438c-8e80-8df6cb5a71d9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.997990 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4178aa8-58af-4be2-bba5-141070f26035 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.020564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1046/2117] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:00.025294 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4178aa8-58af-4be2-bba5-141070f26035 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.044686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4178aa8-58af-4be2-bba5-141070f26035 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.133960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cf644-832d-4e57-8889-f72320245cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.134719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cf644-832d-4e57-8889-f72320245cec admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.135047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cf644-832d-4e57-8889-f72320245cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.135463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cf644-832d-4e57-8889-f72320245cec admin admin] PATH_INFO: `/v3/projects/dd1adf54e9744e09864cee6013128bd9/users/3d85d50149a9494c9753121f7fb03bea/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.137661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fec96-36d5-438c-8e80-8df6cb5a71d9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.153850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50314585-2e8e-4a1e-8273-4c04077397aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.154022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cf644-832d-4e57-8889-f72320245cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd1adf54e9744e09864cee6013128bd9, user_id=3d85d50149a9494c9753121f7fb03bea, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.155061 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50314585-2e8e-4a1e-8273-4c04077397aa admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.155307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50314585-2e8e-4a1e-8273-4c04077397aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.155578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50314585-2e8e-4a1e-8273-4c04077397aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.156256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50314585-2e8e-4a1e-8273-4c04077397aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.157303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cf644-832d-4e57-8889-f72320245cec admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.162791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50314585-2e8e-4a1e-8273-4c04077397aa admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.176286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1047/2118] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:00 2022] PUT /identity/v3/projects/dd1adf54e9744e09864cee6013128bd9/users/3d85d50149a9494c9753121f7fb03bea/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:00.180159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1072/2119] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:00.180857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cf644-832d-4e57-8889-f72320245cec admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.184823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50314585-2e8e-4a1e-8273-4c04077397aa admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.210765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50314585-2e8e-4a1e-8273-4c04077397aa admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.239149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:00.303262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cf644-832d-4e57-8889-f72320245cec admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.309759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0812b-ff45-442a-a702-1fafc7d8f82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.310565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0812b-ff45-442a-a702-1fafc7d8f82f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.310739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0812b-ff45-442a-a702-1fafc7d8f82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.311133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0812b-ff45-442a-a702-1fafc7d8f82f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.311797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0812b-ff45-442a-a702-1fafc7d8f82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.319800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8655820-c484-4c8d-8b5d-96b2d758266b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.319958 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0812b-ff45-442a-a702-1fafc7d8f82f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.321000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8655820-c484-4c8d-8b5d-96b2d758266b admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.321420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8655820-c484-4c8d-8b5d-96b2d758266b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.321853 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8655820-c484-4c8d-8b5d-96b2d758266b admin admin] PATH_INFO: `/v3/projects/22730d1bea7d4b919643a9acfc534ffe/users/f340d1e83b4545dba8dcef466afb6fd5/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.325048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8655820-c484-4c8d-8b5d-96b2d758266b admin admin] RBAC: Authorizing `identity:create_grant(project_id=22730d1bea7d4b919643a9acfc534ffe, user_id=f340d1e83b4545dba8dcef466afb6fd5, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.328252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8655820-c484-4c8d-8b5d-96b2d758266b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.331106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1048/2120] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:00.332022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0812b-ff45-442a-a702-1fafc7d8f82f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.336247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f1f0812b-ff45-442a-a702-1fafc7d8f82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.355922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1073/2121] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:00 2022] PUT /identity/v3/projects/22730d1bea7d4b919643a9acfc534ffe/users/f340d1e83b4545dba8dcef466afb6fd5/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:00.356731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8655820-c484-4c8d-8b5d-96b2d758266b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.369894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0812b-ff45-442a-a702-1fafc7d8f82f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.407829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:00.428340 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8655820-c484-4c8d-8b5d-96b2d758266b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.433068 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725b5df-f5ad-4ba7-b2fe-81d43d19c409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.433764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725b5df-f5ad-4ba7-b2fe-81d43d19c409 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.434043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725b5df-f5ad-4ba7-b2fe-81d43d19c409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.434309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725b5df-f5ad-4ba7-b2fe-81d43d19c409 admin admin] PATH_INFO: `/v3/projects/dd1adf54e9744e09864cee6013128bd9/users/3d85d50149a9494c9753121f7fb03bea/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.434557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.435203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.435520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.435824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] PATH_INFO: `/v3/projects/873a7cd008854368b11cbeda68ebd022` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.436379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725b5df-f5ad-4ba7-b2fe-81d43d19c409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd1adf54e9744e09864cee6013128bd9, user_id=3d85d50149a9494c9753121f7fb03bea, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.436972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=873a7cd008854368b11cbeda68ebd022)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.438904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.439043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725b5df-f5ad-4ba7-b2fe-81d43d19c409 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.449329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1074/2122] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:00 2022] PUT /identity/v3/projects/dd1adf54e9744e09864cee6013128bd9/users/3d85d50149a9494c9753121f7fb03bea/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:00.460691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725b5df-f5ad-4ba7-b2fe-81d43d19c409 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.462766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f725b5df-f5ad-4ba7-b2fe-81d43d19c409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.496665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '873a7cd008854368b11cbeda68ebd022', 'request_id': 'req-3f22bffb-79d6-4be2-b9bb-2ed472115f61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.504859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '873a7cd008854368b11cbeda68ebd022', 'request_id': 'req-3f22bffb-79d6-4be2-b9bb-2ed472115f61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.510943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '873a7cd008854368b11cbeda68ebd022', 'request_id': 'req-3f22bffb-79d6-4be2-b9bb-2ed472115f61'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.526763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] The token cache is being invalidate because project 873a7cd008854368b11cbeda68ebd022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:00.527255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.527906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3f22bffb-79d6-4be2-b9bb-2ed472115f61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.530263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1049/2123] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:00 2022] DELETE /identity/v3/projects/873a7cd008854368b11cbeda68ebd022 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:00.531409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacda4e-1e8a-4713-a9e1-2af4f7a5d4b3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.531734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacda4e-1e8a-4713-a9e1-2af4f7a5d4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.532008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacda4e-1e8a-4713-a9e1-2af4f7a5d4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.542195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725b5df-f5ad-4ba7-b2fe-81d43d19c409 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.566535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7eacda4e-1e8a-4713-a9e1-2af4f7a5d4b3 None None] MFA Rules not processed for user `f340d1e83b4545dba8dcef466afb6fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:00.567724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7eacda4e-1e8a-4713-a9e1-2af4f7a5d4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.606026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:00.619715 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b27ee2-6e0e-47fd-a0dd-7ec31dfc84a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.620393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b27ee2-6e0e-47fd-a0dd-7ec31dfc84a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.620676 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b27ee2-6e0e-47fd-a0dd-7ec31dfc84a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.620955 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b27ee2-6e0e-47fd-a0dd-7ec31dfc84a9 admin admin] PATH_INFO: `/v3/users/a8f723c3f8134cf69eaf43e0210b2139` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.622124 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b27ee2-6e0e-47fd-a0dd-7ec31dfc84a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f723c3f8134cf69eaf43e0210b2139)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.624384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b27ee2-6e0e-47fd-a0dd-7ec31dfc84a9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.670688 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1050/2124] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:00.672061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabad1b-c95a-4a10-8762-be7c6ba14ffd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.672383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabad1b-c95a-4a10-8762-be7c6ba14ffd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.672662 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabad1b-c95a-4a10-8762-be7c6ba14ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.680310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c9b27ee2-6e0e-47fd-a0dd-7ec31dfc84a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f723c3f8134cf69eaf43e0210b2139'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.682210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c9b27ee2-6e0e-47fd-a0dd-7ec31dfc84a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f723c3f8134cf69eaf43e0210b2139'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.683025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c9b27ee2-6e0e-47fd-a0dd-7ec31dfc84a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f723c3f8134cf69eaf43e0210b2139'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.693771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c9b27ee2-6e0e-47fd-a0dd-7ec31dfc84a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f723c3f8134cf69eaf43e0210b2139'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.702407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c9b27ee2-6e0e-47fd-a0dd-7ec31dfc84a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f723c3f8134cf69eaf43e0210b2139'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.705141 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9fabad1b-c95a-4a10-8762-be7c6ba14ffd None None] MFA Rules not processed for user `3d85d50149a9494c9753121f7fb03bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:00.706622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9fabad1b-c95a-4a10-8762-be7c6ba14ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.723584 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1075/2125] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:00 2022] DELETE /identity/v3/users/a8f723c3f8134cf69eaf43e0210b2139 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:00.724645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90b76c-d839-4a14-abb8-51e15d103c13 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.724936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90b76c-d839-4a14-abb8-51e15d103c13 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.725218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90b76c-d839-4a14-abb8-51e15d103c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.746261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fe90b76c-d839-4a14-abb8-51e15d103c13 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:00.747514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fe90b76c-d839-4a14-abb8-51e15d103c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.756571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:00.805396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1051/2126] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:00.806515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabad1b-c95a-4a10-8762-be7c6ba14ffd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.807635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9fabad1b-c95a-4a10-8762-be7c6ba14ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.833298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1076/2127] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:00.834495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf248-1a9c-4583-a829-7e080852ebe8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.834836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf248-1a9c-4583-a829-7e080852ebe8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.835110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf248-1a9c-4583-a829-7e080852ebe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.854563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c89cf248-1a9c-4583-a829-7e080852ebe8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:00.855774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c89cf248-1a9c-4583-a829-7e080852ebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.863779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1077/2128] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:00.867144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cf248-1a9c-4583-a829-7e080852ebe8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.867144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c89cf248-1a9c-4583-a829-7e080852ebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.887953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabad1b-c95a-4a10-8762-be7c6ba14ffd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.947742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cf248-1a9c-4583-a829-7e080852ebe8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.954127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7efc5e-51f5-43e9-85fe-0d9ecbe1ad14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.955457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7efc5e-51f5-43e9-85fe-0d9ecbe1ad14 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.955930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7efc5e-51f5-43e9-85fe-0d9ecbe1ad14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.956441 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7efc5e-51f5-43e9-85fe-0d9ecbe1ad14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.958822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f9d60-7666-4c2f-a4de-e36d7a9ed030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.959415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f9d60-7666-4c2f-a4de-e36d7a9ed030 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.959697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f9d60-7666-4c2f-a4de-e36d7a9ed030 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.960084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f9d60-7666-4c2f-a4de-e36d7a9ed030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.962056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-290f9d60-7666-4c2f-a4de-e36d7a9ed030 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.980681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:01.044065 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7efc5e-51f5-43e9-85fe-0d9ecbe1ad14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.046796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7efc5e-51f5-43e9-85fe-0d9ecbe1ad14 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.048631 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f9d60-7666-4c2f-a4de-e36d7a9ed030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.050455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f9d60-7666-4c2f-a4de-e36d7a9ed030 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.061816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1052/2129] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:01.063135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f0465-43dd-4bef-af78-9c0b13d32cac None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.063483 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f0465-43dd-4bef-af78-9c0b13d32cac None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.063818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f0465-43dd-4bef-af78-9c0b13d32cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.068257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1078/2130] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:01.070372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7efc5e-51f5-43e9-85fe-0d9ecbe1ad14 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.072615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5e7efc5e-51f5-43e9-85fe-0d9ecbe1ad14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:01.135036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c98f0465-43dd-4bef-af78-9c0b13d32cac None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:01.140628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c98f0465-43dd-4bef-af78-9c0b13d32cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:01.161865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1053/2131] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:01.166074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7efc5e-51f5-43e9-85fe-0d9ecbe1ad14 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.167826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f0465-43dd-4bef-af78-9c0b13d32cac None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.171110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107671cd-0cdb-4b97-8b7f-02bde4fc4376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.171110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107671cd-0cdb-4b97-8b7f-02bde4fc4376 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.171110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107671cd-0cdb-4b97-8b7f-02bde4fc4376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.171419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107671cd-0cdb-4b97-8b7f-02bde4fc4376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.194423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f0465-43dd-4bef-af78-9c0b13d32cac None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.202714 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25569073-5dc1-4f86-bd66-2508011855fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.203168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25569073-5dc1-4f86-bd66-2508011855fb admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.203520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25569073-5dc1-4f86-bd66-2508011855fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.203869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25569073-5dc1-4f86-bd66-2508011855fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.204798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25569073-5dc1-4f86-bd66-2508011855fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:01.205012 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107671cd-0cdb-4b97-8b7f-02bde4fc4376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.205308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25569073-5dc1-4f86-bd66-2508011855fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.206505 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107671cd-0cdb-4b97-8b7f-02bde4fc4376 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.207299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25569073-5dc1-4f86-bd66-2508011855fb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.227747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1054/2132] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:01.227747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25569073-5dc1-4f86-bd66-2508011855fb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.237219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1079/2133] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:01.238116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107671cd-0cdb-4b97-8b7f-02bde4fc4376 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.241137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25569073-5dc1-4f86-bd66-2508011855fb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.245217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5c189-ff1a-4d85-9f25-ca21a81fe295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.257886 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5c189-ff1a-4d85-9f25-ca21a81fe295 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.258287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5c189-ff1a-4d85-9f25-ca21a81fe295 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.258575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5c189-ff1a-4d85-9f25-ca21a81fe295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.264605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107671cd-0cdb-4b97-8b7f-02bde4fc4376 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.272377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c2b5c189-ff1a-4d85-9f25-ca21a81fe295 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:01.285540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde18f6-d65a-4694-9469-09cc372d9a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.288980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde18f6-d65a-4694-9469-09cc372d9a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.289364 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde18f6-d65a-4694-9469-09cc372d9a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.289772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde18f6-d65a-4694-9469-09cc372d9a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.290519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde18f6-d65a-4694-9469-09cc372d9a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.292632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde18f6-d65a-4694-9469-09cc372d9a97 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.345676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:01.366037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1080/2134] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:01 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:01.367423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde18f6-d65a-4694-9469-09cc372d9a97 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.422326 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b5c189-ff1a-4d85-9f25-ca21a81fe295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.424178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b5c189-ff1a-4d85-9f25-ca21a81fe295 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.435075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1055/2135] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:01.447509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde18f6-d65a-4694-9469-09cc372d9a97 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.453651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466ff83-4888-4cdd-b739-fdcefac70429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.454371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ff83-4888-4cdd-b739-fdcefac70429 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.454718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ff83-4888-4cdd-b739-fdcefac70429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.455003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ff83-4888-4cdd-b739-fdcefac70429 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.455723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466ff83-4888-4cdd-b739-fdcefac70429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.461376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466ff83-4888-4cdd-b739-fdcefac70429 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.484066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1081/2136] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:01 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:01.488487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5c189-ff1a-4d85-9f25-ca21a81fe295 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.501015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5c189-ff1a-4d85-9f25-ca21a81fe295 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.509289 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe6b74-7cd0-4569-83af-aeda202df967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.509289 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe6b74-7cd0-4569-83af-aeda202df967 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.509550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe6b74-7cd0-4569-83af-aeda202df967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.509974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe6b74-7cd0-4569-83af-aeda202df967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.510702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe6b74-7cd0-4569-83af-aeda202df967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.512499 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe6b74-7cd0-4569-83af-aeda202df967 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.527141 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1056/2137] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:01.528904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466ff83-4888-4cdd-b739-fdcefac70429 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.542577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466ff83-4888-4cdd-b739-fdcefac70429 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.549874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a00969-96a6-4172-b3e7-2562a8da1f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.550578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a00969-96a6-4172-b3e7-2562a8da1f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.551127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a00969-96a6-4172-b3e7-2562a8da1f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.551393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a00969-96a6-4172-b3e7-2562a8da1f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.552519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a00969-96a6-4172-b3e7-2562a8da1f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.555036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a00969-96a6-4172-b3e7-2562a8da1f9e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.574038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1082/2138] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:01.574906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe6b74-7cd0-4569-83af-aeda202df967 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.586873 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe6b74-7cd0-4569-83af-aeda202df967 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.594065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722fc4c5-77fa-4139-85bf-2c4acc951428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.594877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fc4c5-77fa-4139-85bf-2c4acc951428 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.595630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fc4c5-77fa-4139-85bf-2c4acc951428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.595630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fc4c5-77fa-4139-85bf-2c4acc951428 admin admin] PATH_INFO: `/v3/projects/ca73c0a36d394b0fa47c5b7e9b09ef59/users/558feb74daa446e6b3af89bed28622a9/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.629596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722fc4c5-77fa-4139-85bf-2c4acc951428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca73c0a36d394b0fa47c5b7e9b09ef59, user_id=558feb74daa446e6b3af89bed28622a9, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.632461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722fc4c5-77fa-4139-85bf-2c4acc951428 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.658147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a00969-96a6-4172-b3e7-2562a8da1f9e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.661180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e2a00969-96a6-4172-b3e7-2562a8da1f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:01.663743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1057/2139] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:01 2022] PUT /identity/v3/projects/ca73c0a36d394b0fa47c5b7e9b09ef59/users/558feb74daa446e6b3af89bed28622a9/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:01.677316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a00969-96a6-4172-b3e7-2562a8da1f9e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.679053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e21731-e05d-4e63-a7f9-3464d4a09883 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.680328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e21731-e05d-4e63-a7f9-3464d4a09883 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.680623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e21731-e05d-4e63-a7f9-3464d4a09883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.701039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:01.779717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d6e21731-e05d-4e63-a7f9-3464d4a09883 None None] MFA Rules not processed for user `558feb74daa446e6b3af89bed28622a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:01.781594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d6e21731-e05d-4e63-a7f9-3464d4a09883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:01.806821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.809088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.809380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.809663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] PATH_INFO: `/v3/projects/6633acea2962443aa66f3a9d1d91ade2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.812481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6633acea2962443aa66f3a9d1d91ade2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.814774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.904792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6633acea2962443aa66f3a9d1d91ade2', 'request_id': 'req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:01.913349 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6633acea2962443aa66f3a9d1d91ade2', 'request_id': 'req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:01.923098 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6633acea2962443aa66f3a9d1d91ade2', 'request_id': 'req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:01.925423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1058/2140] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:01.930331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e21731-e05d-4e63-a7f9-3464d4a09883 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.945174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] The token cache is being invalidate because project 6633acea2962443aa66f3a9d1d91ade2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:01.945693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:01.947618 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:01.948896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e21731-e05d-4e63-a7f9-3464d4a09883 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.953755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1083/2141] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:01 2022] DELETE /identity/v3/projects/6633acea2962443aa66f3a9d1d91ade2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:01.955404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.962454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:02.055540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f3ab1-36a9-4b42-943a-52d24e58a035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.056432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f3ab1-36a9-4b42-943a-52d24e58a035 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.056831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f3ab1-36a9-4b42-943a-52d24e58a035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.057202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f3ab1-36a9-4b42-943a-52d24e58a035 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.058167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f3ab1-36a9-4b42-943a-52d24e58a035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.061622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f3ab1-36a9-4b42-943a-52d24e58a035 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.087654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac506ed-da9e-4e5c-919d-a1fc4cfe8fa8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:02.095986 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.097339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.097677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.108259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:02.110715 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] PATH_INFO: `/v3/users/afeaa2fd2de4453ca2ae121d62c5beab` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.115660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afeaa2fd2de4453ca2ae121d62c5beab)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.118145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1059/2142] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:01 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:02.124105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f3ab1-36a9-4b42-943a-52d24e58a035 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:02.124731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.126363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b53f3ab1-36a9-4b42-943a-52d24e58a035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:02.218413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afeaa2fd2de4453ca2ae121d62c5beab'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.219228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afeaa2fd2de4453ca2ae121d62c5beab'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.219950 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afeaa2fd2de4453ca2ae121d62c5beab'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.222200 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:02.231559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f3ab1-36a9-4b42-943a-52d24e58a035 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:02.234284 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afeaa2fd2de4453ca2ae121d62c5beab'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.238095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919ab3e-594b-4751-b20b-ccf465be651e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.238706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ab3e-594b-4751-b20b-ccf465be651e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.239071 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ab3e-594b-4751-b20b-ccf465be651e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.239305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ab3e-594b-4751-b20b-ccf465be651e admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.240132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919ab3e-594b-4751-b20b-ccf465be651e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.242438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919ab3e-594b-4751-b20b-ccf465be651e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.244941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afeaa2fd2de4453ca2ae121d62c5beab'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.273307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1060/2143] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:02 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:02.274331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919ab3e-594b-4751-b20b-ccf465be651e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:02.274448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1084/2144] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:01 2022] DELETE /identity/v3/users/afeaa2fd2de4453ca2ae121d62c5beab => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:02.278509 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6919ab3e-594b-4751-b20b-ccf465be651e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:02.292435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:02.294258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:02.316412 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919ab3e-594b-4751-b20b-ccf465be651e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:02.454061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0032e7-1c3f-4696-81db-a3c469d376c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.454822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0032e7-1c3f-4696-81db-a3c469d376c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.455104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0032e7-1c3f-4696-81db-a3c469d376c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.456136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0032e7-1c3f-4696-81db-a3c469d376c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.456482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0032e7-1c3f-4696-81db-a3c469d376c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.458327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0032e7-1c3f-4696-81db-a3c469d376c6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.464230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305f0a-7408-4a79-8a61-f397b0c31323 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:02.470968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49f961-199b-4d8a-9a9d-6568de37b7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.471792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49f961-199b-4d8a-9a9d-6568de37b7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.472176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49f961-199b-4d8a-9a9d-6568de37b7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.472481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49f961-199b-4d8a-9a9d-6568de37b7e3 admin admin] PATH_INFO: `/v3/users/5660fc040b204d66ba1b2e9ce52dfb6e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.473736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49f961-199b-4d8a-9a9d-6568de37b7e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5660fc040b204d66ba1b2e9ce52dfb6e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.474189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1061/2145] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:02.476309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49f961-199b-4d8a-9a9d-6568de37b7e3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.484825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0032e7-1c3f-4696-81db-a3c469d376c6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:02.496489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0032e7-1c3f-4696-81db-a3c469d376c6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:02.507228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d60cd6-6c47-4d09-ba2e-58dbac20a409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.508306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d60cd6-6c47-4d09-ba2e-58dbac20a409 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.508435 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d60cd6-6c47-4d09-ba2e-58dbac20a409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.508824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d60cd6-6c47-4d09-ba2e-58dbac20a409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.510067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d60cd6-6c47-4d09-ba2e-58dbac20a409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.513816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d60cd6-6c47-4d09-ba2e-58dbac20a409 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.540037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1062/2146] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:02.544070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d60cd6-6c47-4d09-ba2e-58dbac20a409 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:02.558171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d60cd6-6c47-4d09-ba2e-58dbac20a409 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:02.563089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3fd48-e25c-47b8-bac1-ba6743e0466f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.563749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3fd48-e25c-47b8-bac1-ba6743e0466f admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.564095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3fd48-e25c-47b8-bac1-ba6743e0466f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.564334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3fd48-e25c-47b8-bac1-ba6743e0466f admin admin] PATH_INFO: `/v3/projects/92ba51632c0646f5aa59afa413da65ec/users/c656be83dbe44671a1d30004d4521332/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.571225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-be49f961-199b-4d8a-9a9d-6568de37b7e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5660fc040b204d66ba1b2e9ce52dfb6e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.572291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-be49f961-199b-4d8a-9a9d-6568de37b7e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5660fc040b204d66ba1b2e9ce52dfb6e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.573152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-be49f961-199b-4d8a-9a9d-6568de37b7e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5660fc040b204d66ba1b2e9ce52dfb6e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.584578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-be49f961-199b-4d8a-9a9d-6568de37b7e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5660fc040b204d66ba1b2e9ce52dfb6e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.596902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-be49f961-199b-4d8a-9a9d-6568de37b7e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5660fc040b204d66ba1b2e9ce52dfb6e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.599180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3fd48-e25c-47b8-bac1-ba6743e0466f admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ba51632c0646f5aa59afa413da65ec, user_id=c656be83dbe44671a1d30004d4521332, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.604798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3fd48-e25c-47b8-bac1-ba6743e0466f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.624244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1063/2147] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:02 2022] PUT /identity/v3/projects/92ba51632c0646f5aa59afa413da65ec/users/c656be83dbe44671a1d30004d4521332/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:02.628967 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1085/2148] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:02 2022] DELETE /identity/v3/users/5660fc040b204d66ba1b2e9ce52dfb6e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:02.635023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a3d75-0e0e-4b59-930a-0268a954944d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.635392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a3d75-0e0e-4b59-930a-0268a954944d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.635739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a3d75-0e0e-4b59-930a-0268a954944d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.642185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977a769-0f6e-4581-8c12-f0f885b85be5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.642516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977a769-0f6e-4581-8c12-f0f885b85be5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.642743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977a769-0f6e-4581-8c12-f0f885b85be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.675929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1d9a3d75-0e0e-4b59-930a-0268a954944d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:02.677470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1d9a3d75-0e0e-4b59-930a-0268a954944d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:02.690639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f977a769-0f6e-4581-8c12-f0f885b85be5 None None] MFA Rules not processed for user `c656be83dbe44671a1d30004d4521332`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:02.692443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f977a769-0f6e-4581-8c12-f0f885b85be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:02.750645 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:02.758799 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:02.782358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1064/2149] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:02.792702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a3d75-0e0e-4b59-930a-0268a954944d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:02.795410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1d9a3d75-0e0e-4b59-930a-0268a954944d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:02.838353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1086/2150] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:02.841039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977a769-0f6e-4581-8c12-f0f885b85be5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:02.843170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f977a769-0f6e-4581-8c12-f0f885b85be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:02.859064 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977a769-0f6e-4581-8c12-f0f885b85be5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:02.864258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.864917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.865198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.865471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] PATH_INFO: `/v3/projects/c28e9eab012244468bdfa3f98ccefb4e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.866924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c28e9eab012244468bdfa3f98ccefb4e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.868741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.873911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a3d75-0e0e-4b59-930a-0268a954944d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:02.877579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdf045-6257-433a-8996-9b7e6c59eaac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.878175 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdf045-6257-433a-8996-9b7e6c59eaac service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.879251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdf045-6257-433a-8996-9b7e6c59eaac service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.879532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdf045-6257-433a-8996-9b7e6c59eaac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.892513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdf045-6257-433a-8996-9b7e6c59eaac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.893958 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdf045-6257-433a-8996-9b7e6c59eaac service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.906166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1065/2151] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:02.907112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdf045-6257-433a-8996-9b7e6c59eaac service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:02.908498 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-facdf045-6257-433a-8996-9b7e6c59eaac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:02.925353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdf045-6257-433a-8996-9b7e6c59eaac service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:02.931318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bdd16-e2b7-4719-bdab-cd68ebc6ca09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.932138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bdd16-e2b7-4719-bdab-cd68ebc6ca09 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.932497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bdd16-e2b7-4719-bdab-cd68ebc6ca09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.932847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bdd16-e2b7-4719-bdab-cd68ebc6ca09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.933663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bdd16-e2b7-4719-bdab-cd68ebc6ca09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.936359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bdd16-e2b7-4719-bdab-cd68ebc6ca09 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.951266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c28e9eab012244468bdfa3f98ccefb4e', 'request_id': 'req-c0a341db-fdf7-44ac-b062-9d248ab0f699'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.962617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c28e9eab012244468bdfa3f98ccefb4e', 'request_id': 'req-c0a341db-fdf7-44ac-b062-9d248ab0f699'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.971840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c28e9eab012244468bdfa3f98ccefb4e', 'request_id': 'req-c0a341db-fdf7-44ac-b062-9d248ab0f699'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:02.977990 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1066/2152] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:02 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:02.983074 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bdd16-e2b7-4719-bdab-cd68ebc6ca09 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:02.999465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] The token cache is being invalidate because project c28e9eab012244468bdfa3f98ccefb4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:02.999633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.001204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c0a341db-fdf7-44ac-b062-9d248ab0f699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.006166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1087/2153] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:02 2022] DELETE /identity/v3/projects/c28e9eab012244468bdfa3f98ccefb4e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:03.012436 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaeb97-c748-4a3e-a087-bc27bf0a439e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.012799 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaeb97-c748-4a3e-a087-bc27bf0a439e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.013114 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaeb97-c748-4a3e-a087-bc27bf0a439e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.045379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-30eaeb97-c748-4a3e-a087-bc27bf0a439e None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:03.046571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-30eaeb97-c748-4a3e-a087-bc27bf0a439e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:03.057138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1088/2154] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:03.057821 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bdd16-e2b7-4719-bdab-cd68ebc6ca09 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.062782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd58c1-eae9-4306-9085-6ef670911fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.063400 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd58c1-eae9-4306-9085-6ef670911fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.063678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd58c1-eae9-4306-9085-6ef670911fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.063957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd58c1-eae9-4306-9085-6ef670911fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.064638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd58c1-eae9-4306-9085-6ef670911fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.064746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d500d1b-099e-443b-a489-4e6e842b2fa6 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.065030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d500d1b-099e-443b-a489-4e6e842b2fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.065308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d500d1b-099e-443b-a489-4e6e842b2fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.066698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd58c1-eae9-4306-9085-6ef670911fc5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.095334 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8d500d1b-099e-443b-a489-4e6e842b2fa6 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:03.097374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8d500d1b-099e-443b-a489-4e6e842b2fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:03.102335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1067/2155] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:02 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:03.107960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd58c1-eae9-4306-9085-6ef670911fc5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.111063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1089/2156] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:03.115069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d500d1b-099e-443b-a489-4e6e842b2fa6 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.115590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7cdd58c1-eae9-4306-9085-6ef670911fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:03.129707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd58c1-eae9-4306-9085-6ef670911fc5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.133219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d500d1b-099e-443b-a489-4e6e842b2fa6 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.135446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add27c1-7bb0-45f3-88fa-307f56a7ba5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.138921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaeee71-e603-4f4b-8c58-9c79571b6340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.139669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaeee71-e603-4f4b-8c58-9c79571b6340 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.140004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaeee71-e603-4f4b-8c58-9c79571b6340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.140396 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaeee71-e603-4f4b-8c58-9c79571b6340 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.141167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaeee71-e603-4f4b-8c58-9c79571b6340 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:03.141504 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaeee71-e603-4f4b-8c58-9c79571b6340 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.143441 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaeee71-e603-4f4b-8c58-9c79571b6340 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.148330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add27c1-7bb0-45f3-88fa-307f56a7ba5f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.148737 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add27c1-7bb0-45f3-88fa-307f56a7ba5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.150670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add27c1-7bb0-45f3-88fa-307f56a7ba5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.151525 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add27c1-7bb0-45f3-88fa-307f56a7ba5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.153082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add27c1-7bb0-45f3-88fa-307f56a7ba5f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.173413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1090/2157] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:03.177845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaeee71-e603-4f4b-8c58-9c79571b6340 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.181202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1068/2158] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:03.184740 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add27c1-7bb0-45f3-88fa-307f56a7ba5f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.198448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaeee71-e603-4f4b-8c58-9c79571b6340 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.201045 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add27c1-7bb0-45f3-88fa-307f56a7ba5f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.206609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f64a23-4785-4955-bf28-d2f4d42df174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.208589 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30457a1f-8ad1-4ae2-b528-2c15c193220b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.209669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30457a1f-8ad1-4ae2-b528-2c15c193220b admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.210157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30457a1f-8ad1-4ae2-b528-2c15c193220b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.210575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30457a1f-8ad1-4ae2-b528-2c15c193220b admin admin] PATH_INFO: `/v3/projects/e909f1c8d76e43569de975fec698bb93/users/299e8544a4694b089508927f51c07307/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.212994 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f64a23-4785-4955-bf28-d2f4d42df174 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.213190 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f64a23-4785-4955-bf28-d2f4d42df174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.215178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f64a23-4785-4955-bf28-d2f4d42df174 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.215933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f64a23-4785-4955-bf28-d2f4d42df174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.218800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f64a23-4785-4955-bf28-d2f4d42df174 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.238901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30457a1f-8ad1-4ae2-b528-2c15c193220b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e909f1c8d76e43569de975fec698bb93, user_id=299e8544a4694b089508927f51c07307, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.247109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30457a1f-8ad1-4ae2-b528-2c15c193220b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.262620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1091/2159] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:03 2022] POST /identity/v3/projects => generated 405 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:03.263964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f64a23-4785-4955-bf28-d2f4d42df174 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.295833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1069/2160] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:03 2022] PUT /identity/v3/projects/e909f1c8d76e43569de975fec698bb93/users/299e8544a4694b089508927f51c07307/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:03.301753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30457a1f-8ad1-4ae2-b528-2c15c193220b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.410946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f64a23-4785-4955-bf28-d2f4d42df174 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.412150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:03.438448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:03.458177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30457a1f-8ad1-4ae2-b528-2c15c193220b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.464670 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e89510-f36f-4d0e-9a5d-f0e0e04630ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.468256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e89510-f36f-4d0e-9a5d-f0e0e04630ac admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.468651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e89510-f36f-4d0e-9a5d-f0e0e04630ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.468755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e89510-f36f-4d0e-9a5d-f0e0e04630ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.469404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e89510-f36f-4d0e-9a5d-f0e0e04630ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.471169 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e89510-f36f-4d0e-9a5d-f0e0e04630ac admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.482372 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1070/2161] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:03.485538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e89510-f36f-4d0e-9a5d-f0e0e04630ac admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.487747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-02e89510-f36f-4d0e-9a5d-f0e0e04630ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:03.497533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ba93b-9baf-4e64-bb98-7a353384abad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.498447 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ba93b-9baf-4e64-bb98-7a353384abad admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.498743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ba93b-9baf-4e64-bb98-7a353384abad admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.499014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ba93b-9baf-4e64-bb98-7a353384abad admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.499657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ba93b-9baf-4e64-bb98-7a353384abad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.501815 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ba93b-9baf-4e64-bb98-7a353384abad admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.503909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e89510-f36f-4d0e-9a5d-f0e0e04630ac admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.518532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.519174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.519456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.519729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] PATH_INFO: `/v3/projects/5f31b79049464a7193d85f1dc03b9f0a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.520906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f31b79049464a7193d85f1dc03b9f0a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.522805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.528148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1092/2162] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:03 2022] POST /identity/v3/users => generated 303 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:03.528148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ba93b-9baf-4e64-bb98-7a353384abad admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.540201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ba93b-9baf-4e64-bb98-7a353384abad admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.545241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69fdd5-7899-4773-bc9d-b8e1466701e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.548394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69fdd5-7899-4773-bc9d-b8e1466701e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.548688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69fdd5-7899-4773-bc9d-b8e1466701e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.548977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69fdd5-7899-4773-bc9d-b8e1466701e3 admin admin] PATH_INFO: `/v3/projects/e909f1c8d76e43569de975fec698bb93/users/299e8544a4694b089508927f51c07307/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.551120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69fdd5-7899-4773-bc9d-b8e1466701e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e909f1c8d76e43569de975fec698bb93, user_id=299e8544a4694b089508927f51c07307, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.554056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69fdd5-7899-4773-bc9d-b8e1466701e3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.568197 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1093/2163] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:03 2022] PUT /identity/v3/projects/e909f1c8d76e43569de975fec698bb93/users/299e8544a4694b089508927f51c07307/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:03.568980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69fdd5-7899-4773-bc9d-b8e1466701e3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.602517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f31b79049464a7193d85f1dc03b9f0a', 'request_id': 'req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.613310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f31b79049464a7193d85f1dc03b9f0a', 'request_id': 'req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.618820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f31b79049464a7193d85f1dc03b9f0a', 'request_id': 'req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.641946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] The token cache is being invalidate because project 5f31b79049464a7193d85f1dc03b9f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:03.642401 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.643131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca3b23f-1290-49c7-a032-dd4c4b6af3e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.645023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1071/2164] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:03 2022] DELETE /identity/v3/projects/5f31b79049464a7193d85f1dc03b9f0a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:03.649216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec92c0e-85a9-46f2-ad3c-6f4ff5c9f711 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.650242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec92c0e-85a9-46f2-ad3c-6f4ff5c9f711 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.650568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec92c0e-85a9-46f2-ad3c-6f4ff5c9f711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.672542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69fdd5-7899-4773-bc9d-b8e1466701e3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.686385 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fec92c0e-85a9-46f2-ad3c-6f4ff5c9f711 None None] MFA Rules not processed for user `299e8544a4694b089508927f51c07307`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:03.689829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fec92c0e-85a9-46f2-ad3c-6f4ff5c9f711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:03.776746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fec85b-6463-4216-9e17-3cc25f97de70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.778465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fec85b-6463-4216-9e17-3cc25f97de70 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.778778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fec85b-6463-4216-9e17-3cc25f97de70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.779088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fec85b-6463-4216-9e17-3cc25f97de70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.779837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fec85b-6463-4216-9e17-3cc25f97de70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.781657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fec85b-6463-4216-9e17-3cc25f97de70 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.796825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1072/2165] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:03.797072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1094/2166] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:03.801173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fec85b-6463-4216-9e17-3cc25f97de70 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.802092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec92c0e-85a9-46f2-ad3c-6f4ff5c9f711 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.803563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fec92c0e-85a9-46f2-ad3c-6f4ff5c9f711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:03.806624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e6fec85b-6463-4216-9e17-3cc25f97de70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:03.835793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fec85b-6463-4216-9e17-3cc25f97de70 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.838072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec92c0e-85a9-46f2-ad3c-6f4ff5c9f711 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:03.842599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.844777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100f65b-1584-422b-b402-b52cd1ca6926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:03.845498 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.845498 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.845675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100f65b-1584-422b-b402-b52cd1ca6926 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:03.845675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100f65b-1584-422b-b402-b52cd1ca6926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:03.845817 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] PATH_INFO: `/v3/users/3d85d50149a9494c9753121f7fb03bea` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.845894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100f65b-1584-422b-b402-b52cd1ca6926 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:03.845894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100f65b-1584-422b-b402-b52cd1ca6926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.846675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100f65b-1584-422b-b402-b52cd1ca6926 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.846879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d85d50149a9494c9753121f7fb03bea)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:03.849077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:03.863916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1095/2167] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:03.865284 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100f65b-1584-422b-b402-b52cd1ca6926 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:03.867917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9100f65b-1584-422b-b402-b52cd1ca6926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:03.909801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:03.929043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:03.947954 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d85d50149a9494c9753121f7fb03bea'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.948742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d85d50149a9494c9753121f7fb03bea'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.952418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d85d50149a9494c9753121f7fb03bea'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.969861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d85d50149a9494c9753121f7fb03bea'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.979682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d85d50149a9494c9753121f7fb03bea'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:03.999306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100f65b-1584-422b-b402-b52cd1ca6926 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:04.004651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2623d8-4c66-46db-a53f-481543b5d3da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:04.005335 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2623d8-4c66-46db-a53f-481543b5d3da service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:04.005621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2623d8-4c66-46db-a53f-481543b5d3da service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:04.005903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2623d8-4c66-46db-a53f-481543b5d3da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:04.007340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1073/2168] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:03 2022] DELETE /identity/v3/users/3d85d50149a9494c9753121f7fb03bea => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:04.008209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:04.009644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:04.012576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ae2623d8-4c66-46db-a53f-481543b5d3da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:04.117771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2623d8-4c66-46db-a53f-481543b5d3da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:04.119585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2623d8-4c66-46db-a53f-481543b5d3da service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:04.123152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de49a96-5d97-4508-90f6-685d90666409 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:04.132361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28be78-08c1-428e-9f6b-dc59bb4052c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:04.133187 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28be78-08c1-428e-9f6b-dc59bb4052c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:04.133444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1096/2169] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:04.136256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28be78-08c1-428e-9f6b-dc59bb4052c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:04.136571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28be78-08c1-428e-9f6b-dc59bb4052c0 admin admin] PATH_INFO: `/v3/projects/f9087a2f9b9347809a53a2c3896cf8d0/users/0e39cf9dbd264e71bacf8a26aa21fb5c/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:04.156167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28be78-08c1-428e-9f6b-dc59bb4052c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9087a2f9b9347809a53a2c3896cf8d0, user_id=0e39cf9dbd264e71bacf8a26aa21fb5c, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:04.158943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28be78-08c1-428e-9f6b-dc59bb4052c0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:04.180835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1074/2170] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:04 2022] PUT /identity/v3/projects/f9087a2f9b9347809a53a2c3896cf8d0/users/0e39cf9dbd264e71bacf8a26aa21fb5c/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:04.195130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e415ed3-22f2-4fe9-a2d9-279c1df4fb9e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:04.196030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e415ed3-22f2-4fe9-a2d9-279c1df4fb9e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:04.196623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e415ed3-22f2-4fe9-a2d9-279c1df4fb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:04.245649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2e415ed3-22f2-4fe9-a2d9-279c1df4fb9e None None] MFA Rules not processed for user `0e39cf9dbd264e71bacf8a26aa21fb5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:04.246976 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2e415ed3-22f2-4fe9-a2d9-279c1df4fb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:04.332164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1097/2171] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:04 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:04.333540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28be78-08c1-428e-9f6b-dc59bb4052c0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:04.440789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28be78-08c1-428e-9f6b-dc59bb4052c0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:04.454445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc9825-ae93-402e-8c7a-c7206ed7c271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:04.456962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc9825-ae93-402e-8c7a-c7206ed7c271 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:04.457844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc9825-ae93-402e-8c7a-c7206ed7c271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:04.458397 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc9825-ae93-402e-8c7a-c7206ed7c271 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:04.461019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc9825-ae93-402e-8c7a-c7206ed7c271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:04.465565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc9825-ae93-402e-8c7a-c7206ed7c271 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:04.517377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1075/2172] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:04 2022] POST /identity/v3/projects => generated 409 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:04.520518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e415ed3-22f2-4fe9-a2d9-279c1df4fb9e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:04.546703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e415ed3-22f2-4fe9-a2d9-279c1df4fb9e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:04.633555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986958e1-999e-4a4f-85a6-d9f907b7ab0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:04.634620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986958e1-999e-4a4f-85a6-d9f907b7ab0e admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:04.635148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986958e1-999e-4a4f-85a6-d9f907b7ab0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:04.636696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986958e1-999e-4a4f-85a6-d9f907b7ab0e admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:04.637560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986958e1-999e-4a4f-85a6-d9f907b7ab0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:04.640208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986958e1-999e-4a4f-85a6-d9f907b7ab0e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:04.659366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:04.669275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1098/2173] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:04 2022] POST /identity/v3/users => generated 311 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:04.673578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc9825-ae93-402e-8c7a-c7206ed7c271 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:04.686632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc9825-ae93-402e-8c7a-c7206ed7c271 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:04.706141 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161407cb-257c-45b4-b7c2-5d604145b19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:04.706767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161407cb-257c-45b4-b7c2-5d604145b19e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:04.707305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161407cb-257c-45b4-b7c2-5d604145b19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:04.707583 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161407cb-257c-45b4-b7c2-5d604145b19e admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:04.708286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161407cb-257c-45b4-b7c2-5d604145b19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:04.709957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161407cb-257c-45b4-b7c2-5d604145b19e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:04.716093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:04.727723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1076/2174] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:04.728108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986958e1-999e-4a4f-85a6-d9f907b7ab0e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:04.744348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986958e1-999e-4a4f-85a6-d9f907b7ab0e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:04.752279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e94db-bfd7-4517-b07c-3e48c81d3327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:04.752939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e94db-bfd7-4517-b07c-3e48c81d3327 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:04.753356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e94db-bfd7-4517-b07c-3e48c81d3327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:04.753634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e94db-bfd7-4517-b07c-3e48c81d3327 admin admin] PATH_INFO: `/v3/projects/55c41585636c4629a92859a2a23c3432/users/a481a88bef9e43ed85c81f7837c260a2/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:04.782860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e94db-bfd7-4517-b07c-3e48c81d3327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c41585636c4629a92859a2a23c3432, user_id=a481a88bef9e43ed85c81f7837c260a2, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:04.785853 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e94db-bfd7-4517-b07c-3e48c81d3327 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:04.804610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1099/2175] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:04 2022] PUT /identity/v3/projects/55c41585636c4629a92859a2a23c3432/users/a481a88bef9e43ed85c81f7837c260a2/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:04.808710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161407cb-257c-45b4-b7c2-5d604145b19e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:04.822124 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161407cb-257c-45b4-b7c2-5d604145b19e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:04.916303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e94db-bfd7-4517-b07c-3e48c81d3327 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:04.918288 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-323e94db-bfd7-4517-b07c-3e48c81d3327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:04.927933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569ffb2-2226-4b8f-aecb-c81805f823a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:04.928585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ffb2-2226-4b8f-aecb-c81805f823a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:04.929065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ffb2-2226-4b8f-aecb-c81805f823a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:04.929259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ffb2-2226-4b8f-aecb-c81805f823a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:04.930938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569ffb2-2226-4b8f-aecb-c81805f823a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:04.932586 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569ffb2-2226-4b8f-aecb-c81805f823a2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:04.964909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1077/2176] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:04.968669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569ffb2-2226-4b8f-aecb-c81805f823a2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:04.987939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569ffb2-2226-4b8f-aecb-c81805f823a2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:04.996999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b023a-2afd-496a-922c-7d748024aee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:04.997796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b023a-2afd-496a-922c-7d748024aee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:04.998167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b023a-2afd-496a-922c-7d748024aee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:04.998577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b023a-2afd-496a-922c-7d748024aee0 admin admin] PATH_INFO: `/v3/projects/55c41585636c4629a92859a2a23c3432/users/a481a88bef9e43ed85c81f7837c260a2/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.000765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b023a-2afd-496a-922c-7d748024aee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c41585636c4629a92859a2a23c3432, user_id=a481a88bef9e43ed85c81f7837c260a2, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:05.004749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b023a-2afd-496a-922c-7d748024aee0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:05.028567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1078/2177] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:04 2022] PUT /identity/v3/projects/55c41585636c4629a92859a2a23c3432/users/a481a88bef9e43ed85c81f7837c260a2/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:05.041640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab78a0b-0fc8-49e4-8b6a-bc04029ebe9d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.041927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab78a0b-0fc8-49e4-8b6a-bc04029ebe9d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.042245 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab78a0b-0fc8-49e4-8b6a-bc04029ebe9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.044513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e94db-bfd7-4517-b07c-3e48c81d3327 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:05.093573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0ab78a0b-0fc8-49e4-8b6a-bc04029ebe9d None None] MFA Rules not processed for user `a481a88bef9e43ed85c81f7837c260a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:05.094840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0ab78a0b-0fc8-49e4-8b6a-bc04029ebe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:05.133556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:05.134291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.134638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.134909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] PATH_INFO: `/v3/projects/dd1adf54e9744e09864cee6013128bd9` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.136092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd1adf54e9744e09864cee6013128bd9)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:05.138153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:05.181303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1079/2178] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:05.182575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab78a0b-0fc8-49e4-8b6a-bc04029ebe9d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:05.184305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0ab78a0b-0fc8-49e4-8b6a-bc04029ebe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:05.192619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd1adf54e9744e09864cee6013128bd9', 'request_id': 'req-4aae7e75-a458-423e-8eb7-9cf81e451bfc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:05.199866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd1adf54e9744e09864cee6013128bd9', 'request_id': 'req-4aae7e75-a458-423e-8eb7-9cf81e451bfc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:05.203087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab78a0b-0fc8-49e4-8b6a-bc04029ebe9d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:05.207216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd1adf54e9744e09864cee6013128bd9', 'request_id': 'req-4aae7e75-a458-423e-8eb7-9cf81e451bfc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:05.224966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] The token cache is being invalidate because project dd1adf54e9744e09864cee6013128bd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:05.225838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:05.226658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:05.228525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1100/2179] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:04 2022] DELETE /identity/v3/projects/dd1adf54e9744e09864cee6013128bd9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:05.229371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:05.232833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:05.249302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:05.282168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02853a7-a8b6-47e7-9540-01bb1fc7cc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:05.283166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02853a7-a8b6-47e7-9540-01bb1fc7cc44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.283444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02853a7-a8b6-47e7-9540-01bb1fc7cc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.283882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02853a7-a8b6-47e7-9540-01bb1fc7cc44 admin admin] PATH_INFO: `/v3/users/8e04acfc2e4340f8afcc8ce8ffb4c983` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.284921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02853a7-a8b6-47e7-9540-01bb1fc7cc44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e04acfc2e4340f8afcc8ce8ffb4c983)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:05.287037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02853a7-a8b6-47e7-9540-01bb1fc7cc44 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:05.298318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:05.298619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae7e75-a458-423e-8eb7-9cf81e451bfc admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:05.303009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92836085-8f4e-48ed-b99e-517029adb2ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:05.303783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92836085-8f4e-48ed-b99e-517029adb2ee service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.304161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92836085-8f4e-48ed-b99e-517029adb2ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.304539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92836085-8f4e-48ed-b99e-517029adb2ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.306727 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-92836085-8f4e-48ed-b99e-517029adb2ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:05.348006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e02853a7-a8b6-47e7-9540-01bb1fc7cc44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e04acfc2e4340f8afcc8ce8ffb4c983'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:05.348862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e02853a7-a8b6-47e7-9540-01bb1fc7cc44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e04acfc2e4340f8afcc8ce8ffb4c983'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:05.349473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e02853a7-a8b6-47e7-9540-01bb1fc7cc44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e04acfc2e4340f8afcc8ce8ffb4c983'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:05.358895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e02853a7-a8b6-47e7-9540-01bb1fc7cc44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e04acfc2e4340f8afcc8ce8ffb4c983'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:05.368557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e02853a7-a8b6-47e7-9540-01bb1fc7cc44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e04acfc2e4340f8afcc8ce8ffb4c983'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:05.395656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92836085-8f4e-48ed-b99e-517029adb2ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:05.397533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92836085-8f4e-48ed-b99e-517029adb2ee service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:05.404826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1080/2180] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:05 2022] DELETE /identity/v3/users/8e04acfc2e4340f8afcc8ce8ffb4c983 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:05.406200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233c9f6-d221-4d95-854c-24d18d0a97a8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.406485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233c9f6-d221-4d95-854c-24d18d0a97a8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.406759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233c9f6-d221-4d95-854c-24d18d0a97a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.410675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1101/2181] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:05.417507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e3419-9725-4009-938a-4f1684e98ee0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.417949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e3419-9725-4009-938a-4f1684e98ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.418231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e3419-9725-4009-938a-4f1684e98ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.432777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e233c9f6-d221-4d95-854c-24d18d0a97a8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:05.434247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e233c9f6-d221-4d95-854c-24d18d0a97a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:05.444149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-346e3419-9725-4009-938a-4f1684e98ee0 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:05.445696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-346e3419-9725-4009-938a-4f1684e98ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:05.513064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1081/2182] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:05.521058 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1102/2183] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:05.521894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e400c5e-622c-4eab-a5e1-0471ed1aba58 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.522166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e400c5e-622c-4eab-a5e1-0471ed1aba58 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.522445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e400c5e-622c-4eab-a5e1-0471ed1aba58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.531458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e3419-9725-4009-938a-4f1684e98ee0 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:05.532905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-346e3419-9725-4009-938a-4f1684e98ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:05.573382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3e400c5e-622c-4eab-a5e1-0471ed1aba58 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:05.575307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3e400c5e-622c-4eab-a5e1-0471ed1aba58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:05.595432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1082/2184] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:05.595432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e400c5e-622c-4eab-a5e1-0471ed1aba58 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:05.599323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3e400c5e-622c-4eab-a5e1-0471ed1aba58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:05.618477 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e3419-9725-4009-938a-4f1684e98ee0 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:05.633369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e841e7-e904-4d07-aed2-8c12ca72da9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:05.634134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e841e7-e904-4d07-aed2-8c12ca72da9b service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.634420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e841e7-e904-4d07-aed2-8c12ca72da9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.635188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e841e7-e904-4d07-aed2-8c12ca72da9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.649352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e841e7-e904-4d07-aed2-8c12ca72da9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:05.651268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e841e7-e904-4d07-aed2-8c12ca72da9b service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:05.666070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1103/2185] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:04:05.667216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e841e7-e904-4d07-aed2-8c12ca72da9b service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:05.681569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e841e7-e904-4d07-aed2-8c12ca72da9b service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:05.683690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:05.690900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439eb9c-2bc3-436d-a907-d005e062157b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:05.692571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439eb9c-2bc3-436d-a907-d005e062157b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.693216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439eb9c-2bc3-436d-a907-d005e062157b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.693699 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439eb9c-2bc3-436d-a907-d005e062157b admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.695559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439eb9c-2bc3-436d-a907-d005e062157b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:05.696073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439eb9c-2bc3-436d-a907-d005e062157b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:05.698468 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439eb9c-2bc3-436d-a907-d005e062157b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:05.716336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1104/2186] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:05.718144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439eb9c-2bc3-436d-a907-d005e062157b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:05.729234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e400c5e-622c-4eab-a5e1-0471ed1aba58 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:05.733943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439eb9c-2bc3-436d-a907-d005e062157b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:05.734198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02458e-65de-4a1b-a768-a5a6739db498 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:05.735278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02458e-65de-4a1b-a768-a5a6739db498 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.735610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02458e-65de-4a1b-a768-a5a6739db498 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.735889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02458e-65de-4a1b-a768-a5a6739db498 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.741662 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4acc9-576f-4f65-971a-ef296f6f7dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:05.742451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4acc9-576f-4f65-971a-ef296f6f7dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.743961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4acc9-576f-4f65-971a-ef296f6f7dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.745017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4acc9-576f-4f65-971a-ef296f6f7dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.746721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4acc9-576f-4f65-971a-ef296f6f7dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:05.748004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-db02458e-65de-4a1b-a768-a5a6739db498 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:05.749712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4acc9-576f-4f65-971a-ef296f6f7dd7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:05.829490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1105/2187] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:05 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:05.830711 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4acc9-576f-4f65-971a-ef296f6f7dd7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:05.841118 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:05.904902 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02458e-65de-4a1b-a768-a5a6739db498 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:05.906425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02458e-65de-4a1b-a768-a5a6739db498 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:05.923891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4acc9-576f-4f65-971a-ef296f6f7dd7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:05.925554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1083/2188] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:05.926412 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02458e-65de-4a1b-a768-a5a6739db498 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:05.928264 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-db02458e-65de-4a1b-a768-a5a6739db498 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:05.938401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677eacd-2031-4462-b4e2-d0a9d8d2c974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:05.939083 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677eacd-2031-4462-b4e2-d0a9d8d2c974 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:05.939507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677eacd-2031-4462-b4e2-d0a9d8d2c974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:05.939786 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677eacd-2031-4462-b4e2-d0a9d8d2c974 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:05.941332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677eacd-2031-4462-b4e2-d0a9d8d2c974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:05.946157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677eacd-2031-4462-b4e2-d0a9d8d2c974 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:05.948755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02458e-65de-4a1b-a768-a5a6739db498 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:05.992077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1106/2189] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:05 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:05.998777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677eacd-2031-4462-b4e2-d0a9d8d2c974 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:06.017696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677eacd-2031-4462-b4e2-d0a9d8d2c974 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:06.027191 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a160317-4d03-4125-8a11-fc945fee75ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:06.028131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a160317-4d03-4125-8a11-fc945fee75ce admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.028519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a160317-4d03-4125-8a11-fc945fee75ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.029115 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a160317-4d03-4125-8a11-fc945fee75ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.030193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a160317-4d03-4125-8a11-fc945fee75ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:06.030378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84233e4-a553-40ef-aad0-e02b600d1e9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:06.030991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84233e4-a553-40ef-aad0-e02b600d1e9c service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.031271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84233e4-a553-40ef-aad0-e02b600d1e9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.031544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84233e4-a553-40ef-aad0-e02b600d1e9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.033616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c84233e4-a553-40ef-aad0-e02b600d1e9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:06.034115 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a160317-4d03-4125-8a11-fc945fee75ce admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:06.056493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1107/2190] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:06.059289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a160317-4d03-4125-8a11-fc945fee75ce admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:06.090859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a160317-4d03-4125-8a11-fc945fee75ce admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:06.098844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5378-2784-4c62-930d-13561e0cbc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:06.099838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5378-2784-4c62-930d-13561e0cbc74 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.100695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5378-2784-4c62-930d-13561e0cbc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.100695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5378-2784-4c62-930d-13561e0cbc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.101742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5378-2784-4c62-930d-13561e0cbc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:06.103897 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5378-2784-4c62-930d-13561e0cbc74 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:06.111350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84233e4-a553-40ef-aad0-e02b600d1e9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:06.112940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84233e4-a553-40ef-aad0-e02b600d1e9c service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:06.125977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1108/2191] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:06.127583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5378-2784-4c62-930d-13561e0cbc74 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:06.129492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1084/2192] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:06.156739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5378-2784-4c62-930d-13561e0cbc74 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:06.163098 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2304a44-25f1-4307-a6de-8b681f9a91bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:06.163871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2304a44-25f1-4307-a6de-8b681f9a91bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.164414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2304a44-25f1-4307-a6de-8b681f9a91bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.164525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2304a44-25f1-4307-a6de-8b681f9a91bb admin admin] PATH_INFO: `/v3/projects/e75f6db00cf340788a011d7daee52d74/users/ebeca2571bac4b779e57a49eb1de5a60/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.187899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2304a44-25f1-4307-a6de-8b681f9a91bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e75f6db00cf340788a011d7daee52d74, user_id=ebeca2571bac4b779e57a49eb1de5a60, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:06.191514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2304a44-25f1-4307-a6de-8b681f9a91bb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:06.210458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1109/2193] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:06 2022] PUT /identity/v3/projects/e75f6db00cf340788a011d7daee52d74/users/ebeca2571bac4b779e57a49eb1de5a60/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:06.222333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7e401-6ce1-41f2-a885-ceffa69d92a4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.222709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7e401-6ce1-41f2-a885-ceffa69d92a4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.222904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7e401-6ce1-41f2-a885-ceffa69d92a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.264648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-10a7e401-6ce1-41f2-a885-ceffa69d92a4 None None] MFA Rules not processed for user `ebeca2571bac4b779e57a49eb1de5a60`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:06.266719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-10a7e401-6ce1-41f2-a885-ceffa69d92a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:06.432414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1085/2194] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:06.436561 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2304a44-25f1-4307-a6de-8b681f9a91bb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:06.535530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7e401-6ce1-41f2-a885-ceffa69d92a4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:06.535530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-10a7e401-6ce1-41f2-a885-ceffa69d92a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:06.577976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:06.598449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2304a44-25f1-4307-a6de-8b681f9a91bb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:06.612512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b7897-b25b-4ed8-a07d-27d108a081a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:06.613336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b7897-b25b-4ed8-a07d-27d108a081a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.613825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b7897-b25b-4ed8-a07d-27d108a081a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.614284 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b7897-b25b-4ed8-a07d-27d108a081a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.615178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b7897-b25b-4ed8-a07d-27d108a081a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:06.619812 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b7897-b25b-4ed8-a07d-27d108a081a8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:06.677586 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7e401-6ce1-41f2-a885-ceffa69d92a4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:06.679502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:06.688772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:06.690430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.690430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.690430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] PATH_INFO: `/v3/projects/b7f27993d4264fbeaba7d1888ada9599` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.690694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1110/2195] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:06 2022] POST /identity/v3/projects => generated 383 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:06.691194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f27993d4264fbeaba7d1888ada9599)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:06.694432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:06.694617 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b7897-b25b-4ed8-a07d-27d108a081a8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:06.782320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b7897-b25b-4ed8-a07d-27d108a081a8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:06.812172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f27993d4264fbeaba7d1888ada9599', 'request_id': 'req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:06.819246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f27993d4264fbeaba7d1888ada9599', 'request_id': 'req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:06.825330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f27993d4264fbeaba7d1888ada9599', 'request_id': 'req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:06.841044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] The token cache is being invalidate because project b7f27993d4264fbeaba7d1888ada9599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:06.842044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:06.842847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:06.845445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1086/2196] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:06 2022] DELETE /identity/v3/projects/b7f27993d4264fbeaba7d1888ada9599 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:06.848535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:06.850894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:06.869577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc6b09-0cdc-4431-a048-75d836f6c21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:06.870207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6b09-0cdc-4431-a048-75d836f6c21c admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.870593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6b09-0cdc-4431-a048-75d836f6c21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.870899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6b09-0cdc-4431-a048-75d836f6c21c admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.871765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc6b09-0cdc-4431-a048-75d836f6c21c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:06.874533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc6b09-0cdc-4431-a048-75d836f6c21c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:06.909683 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1111/2197] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:06 2022] POST /identity/v3/users => generated 298 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:06.913444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc6b09-0cdc-4431-a048-75d836f6c21c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:06.915427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bdbc6b09-0cdc-4431-a048-75d836f6c21c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:06.929697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc6b09-0cdc-4431-a048-75d836f6c21c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:06.934751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc38096-a64b-4cc4-ade1-96a3a5b6d182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:06.935688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc38096-a64b-4cc4-ade1-96a3a5b6d182 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.935977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc38096-a64b-4cc4-ade1-96a3a5b6d182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.936345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc38096-a64b-4cc4-ade1-96a3a5b6d182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.937125 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc38096-a64b-4cc4-ade1-96a3a5b6d182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:06.938811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc38096-a64b-4cc4-ade1-96a3a5b6d182 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:06.941705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9e3d3-716e-4f8e-a624-fb9c9d88ac06 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:06.947075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:06.948008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.949603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.949895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] PATH_INFO: `/v3/users/eaae66d2577b43ebaf9d3cf257559bf7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.951121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaae66d2577b43ebaf9d3cf257559bf7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:06.953083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:06.960164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1112/2198] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:06.963430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc38096-a64b-4cc4-ade1-96a3a5b6d182 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:06.984889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc38096-a64b-4cc4-ade1-96a3a5b6d182 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:07.001659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d984a-6cd7-4081-a1b6-997316d35c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:07.008991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d984a-6cd7-4081-a1b6-997316d35c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:07.009277 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d984a-6cd7-4081-a1b6-997316d35c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:07.009556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d984a-6cd7-4081-a1b6-997316d35c67 admin admin] PATH_INFO: `/v3/projects/742981a8e4df47c7b78e0ebf00fdec5e/users/91a72ae9525244abaf8f760938c4f3b5/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:07.026844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d984a-6cd7-4081-a1b6-997316d35c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=742981a8e4df47c7b78e0ebf00fdec5e, user_id=91a72ae9525244abaf8f760938c4f3b5, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:07.030202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d984a-6cd7-4081-a1b6-997316d35c67 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:07.046537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaae66d2577b43ebaf9d3cf257559bf7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.048674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaae66d2577b43ebaf9d3cf257559bf7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.052322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1113/2199] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:06 2022] PUT /identity/v3/projects/742981a8e4df47c7b78e0ebf00fdec5e/users/91a72ae9525244abaf8f760938c4f3b5/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:07.052466 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaae66d2577b43ebaf9d3cf257559bf7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.055906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d984a-6cd7-4081-a1b6-997316d35c67 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:07.056731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:07.059854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d41d984a-6cd7-4081-a1b6-997316d35c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:07.064171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaae66d2577b43ebaf9d3cf257559bf7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.076749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaae66d2577b43ebaf9d3cf257559bf7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.104946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1087/2200] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:06 2022] DELETE /identity/v3/users/eaae66d2577b43ebaf9d3cf257559bf7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:07.161588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d984a-6cd7-4081-a1b6-997316d35c67 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:07.168505 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:07.264607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb1604-2b7e-420b-be0e-a29c9bc19efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:07.265231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb1604-2b7e-420b-be0e-a29c9bc19efd admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:07.265507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb1604-2b7e-420b-be0e-a29c9bc19efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:07.265776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb1604-2b7e-420b-be0e-a29c9bc19efd admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:07.266436 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb1604-2b7e-420b-be0e-a29c9bc19efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:07.269135 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb1604-2b7e-420b-be0e-a29c9bc19efd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:07.286693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1114/2201] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:07.290744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:07.306320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fa3b9-c300-4646-83bc-ab2c79db418d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:07.314042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a42cf-42d4-4629-87bf-0b8fb1be9833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:07.314820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a42cf-42d4-4629-87bf-0b8fb1be9833 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:07.315047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a42cf-42d4-4629-87bf-0b8fb1be9833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:07.315399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a42cf-42d4-4629-87bf-0b8fb1be9833 admin admin] PATH_INFO: `/v3/projects/742981a8e4df47c7b78e0ebf00fdec5e/users/91a72ae9525244abaf8f760938c4f3b5/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:07.317852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a42cf-42d4-4629-87bf-0b8fb1be9833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=742981a8e4df47c7b78e0ebf00fdec5e, user_id=91a72ae9525244abaf8f760938c4f3b5, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:07.322941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a42cf-42d4-4629-87bf-0b8fb1be9833 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:07.339343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1088/2202] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:07 2022] PUT /identity/v3/projects/742981a8e4df47c7b78e0ebf00fdec5e/users/91a72ae9525244abaf8f760938c4f3b5/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:07.351966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7efbf-b808-4399-9a12-95c63dad4e9a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:07.352425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7efbf-b808-4399-9a12-95c63dad4e9a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:07.352571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7efbf-b808-4399-9a12-95c63dad4e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:07.411825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-62f7efbf-b808-4399-9a12-95c63dad4e9a None None] MFA Rules not processed for user `91a72ae9525244abaf8f760938c4f3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:07.412861 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-62f7efbf-b808-4399-9a12-95c63dad4e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:07.491868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a42cf-42d4-4629-87bf-0b8fb1be9833 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:07.494318 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2e1a42cf-42d4-4629-87bf-0b8fb1be9833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:07.605375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a42cf-42d4-4629-87bf-0b8fb1be9833 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:07.615388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:07.615388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:07.616258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:07.616610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] PATH_INFO: `/v3/projects/aba80db304d146ab94932249d1515e72` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:07.618007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba80db304d146ab94932249d1515e72)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:07.620179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:07.634349 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1115/2203] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:07.635286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7efbf-b808-4399-9a12-95c63dad4e9a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:07.637115 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-62f7efbf-b808-4399-9a12-95c63dad4e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:07.691944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba80db304d146ab94932249d1515e72', 'request_id': 'req-723aaeb0-464e-42fc-8ae7-b1d444c48974'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.701574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba80db304d146ab94932249d1515e72', 'request_id': 'req-723aaeb0-464e-42fc-8ae7-b1d444c48974'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.710420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba80db304d146ab94932249d1515e72', 'request_id': 'req-723aaeb0-464e-42fc-8ae7-b1d444c48974'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.716922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7efbf-b808-4399-9a12-95c63dad4e9a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:07.725397 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] The token cache is being invalidate because project aba80db304d146ab94932249d1515e72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:07.725712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.726457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.728718 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1089/2204] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:07 2022] DELETE /identity/v3/projects/aba80db304d146ab94932249d1515e72 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:07.732972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:07.734695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:07.771915 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:07.778522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3654943-37ca-4aff-8314-efa0fa5e502b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:07.779795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3654943-37ca-4aff-8314-efa0fa5e502b service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:07.780215 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3654943-37ca-4aff-8314-efa0fa5e502b service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:07.780586 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3654943-37ca-4aff-8314-efa0fa5e502b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:07.783071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e3654943-37ca-4aff-8314-efa0fa5e502b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:07.810391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:07.816112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723aaeb0-464e-42fc-8ae7-b1d444c48974 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:07.823281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859379b1-777a-422a-93fe-5469a4ac0136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:07.823991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859379b1-777a-422a-93fe-5469a4ac0136 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:07.824413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859379b1-777a-422a-93fe-5469a4ac0136 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:07.824763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859379b1-777a-422a-93fe-5469a4ac0136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:07.828022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-859379b1-777a-422a-93fe-5469a4ac0136 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:07.878604 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3654943-37ca-4aff-8314-efa0fa5e502b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:07.880590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3654943-37ca-4aff-8314-efa0fa5e502b service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:07.894269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1116/2205] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:07.895216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3654943-37ca-4aff-8314-efa0fa5e502b service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:07.897118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e3654943-37ca-4aff-8314-efa0fa5e502b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:07.901581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859379b1-777a-422a-93fe-5469a4ac0136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:07.903442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859379b1-777a-422a-93fe-5469a4ac0136 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:07.929019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1090/2206] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:07.981688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3654943-37ca-4aff-8314-efa0fa5e502b service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:07.989769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e33f4-9cc1-4b7d-9241-2057f23d1763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:07.990412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e33f4-9cc1-4b7d-9241-2057f23d1763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:07.990697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e33f4-9cc1-4b7d-9241-2057f23d1763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:07.991019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e33f4-9cc1-4b7d-9241-2057f23d1763 admin admin] PATH_INFO: `/v3/users/f340d1e83b4545dba8dcef466afb6fd5` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:07.993002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e33f4-9cc1-4b7d-9241-2057f23d1763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f340d1e83b4545dba8dcef466afb6fd5)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:07.996757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e33f4-9cc1-4b7d-9241-2057f23d1763 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:08.088434 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-939e33f4-9cc1-4b7d-9241-2057f23d1763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f340d1e83b4545dba8dcef466afb6fd5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:08.090444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-939e33f4-9cc1-4b7d-9241-2057f23d1763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f340d1e83b4545dba8dcef466afb6fd5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:08.093090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-939e33f4-9cc1-4b7d-9241-2057f23d1763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f340d1e83b4545dba8dcef466afb6fd5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:08.108377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-939e33f4-9cc1-4b7d-9241-2057f23d1763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f340d1e83b4545dba8dcef466afb6fd5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:08.120596 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-939e33f4-9cc1-4b7d-9241-2057f23d1763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f340d1e83b4545dba8dcef466afb6fd5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:08.151752 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1117/2207] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:07 2022] DELETE /identity/v3/users/f340d1e83b4545dba8dcef466afb6fd5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:09.160753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859379b1-777a-422a-93fe-5469a4ac0136 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.184682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859379b1-777a-422a-93fe-5469a4ac0136 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.306554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:09.307356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.307644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.307953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] PATH_INFO: `/v3/projects/22730d1bea7d4b919643a9acfc534ffe` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.309649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22730d1bea7d4b919643a9acfc534ffe)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:09.312440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:09.380941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22730d1bea7d4b919643a9acfc534ffe', 'request_id': 'req-831af98b-5df0-41c1-805a-47a7822edfc9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:09.390672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22730d1bea7d4b919643a9acfc534ffe', 'request_id': 'req-831af98b-5df0-41c1-805a-47a7822edfc9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:09.420896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22730d1bea7d4b919643a9acfc534ffe', 'request_id': 'req-831af98b-5df0-41c1-805a-47a7822edfc9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:09.447127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] The token cache is being invalidate because project 22730d1bea7d4b919643a9acfc534ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:09.447430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:09.449767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-831af98b-5df0-41c1-805a-47a7822edfc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:09.457383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1091/2208] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:09 2022] DELETE /identity/v3/projects/22730d1bea7d4b919643a9acfc534ffe => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:09.461313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a5076-d89f-4611-8ab5-1b26abf73198 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.462806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a5076-d89f-4611-8ab5-1b26abf73198 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.463325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a5076-d89f-4611-8ab5-1b26abf73198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.492891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ee0a5076-d89f-4611-8ab5-1b26abf73198 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:09.495649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ee0a5076-d89f-4611-8ab5-1b26abf73198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:09.509111 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:09.610566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1118/2209] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:09.621435 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeca831-b10a-4ea6-9bd6-29d802f5b638 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.621910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeca831-b10a-4ea6-9bd6-29d802f5b638 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.622276 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeca831-b10a-4ea6-9bd6-29d802f5b638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.651813 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-dfeca831-b10a-4ea6-9bd6-29d802f5b638 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:09.653090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dfeca831-b10a-4ea6-9bd6-29d802f5b638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:09.671162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a5076-d89f-4611-8ab5-1b26abf73198 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.677164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1092/2210] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:09.704729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a5076-d89f-4611-8ab5-1b26abf73198 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.711304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e69548-4be6-44e1-b700-b30b29656019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:09.711917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e69548-4be6-44e1-b700-b30b29656019 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.712227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e69548-4be6-44e1-b700-b30b29656019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.712496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e69548-4be6-44e1-b700-b30b29656019 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.713368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e69548-4be6-44e1-b700-b30b29656019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:09.714984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e69548-4be6-44e1-b700-b30b29656019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:09.718575 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e69548-4be6-44e1-b700-b30b29656019 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:09.739422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeca831-b10a-4ea6-9bd6-29d802f5b638 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.740116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1119/2211] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:09.757564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeca831-b10a-4ea6-9bd6-29d802f5b638 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.765645 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bc0a0-29f8-4f68-afab-b2ea7f9ed63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:09.768423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bc0a0-29f8-4f68-afab-b2ea7f9ed63e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.768798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bc0a0-29f8-4f68-afab-b2ea7f9ed63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.769210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bc0a0-29f8-4f68-afab-b2ea7f9ed63e admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.770012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bc0a0-29f8-4f68-afab-b2ea7f9ed63e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:09.773263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bc0a0-29f8-4f68-afab-b2ea7f9ed63e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:09.814648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1093/2212] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:09 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:09.815518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e69548-4be6-44e1-b700-b30b29656019 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.831513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e69548-4be6-44e1-b700-b30b29656019 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.866617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3246ad-7dc3-4491-94ff-5daf3b286a26 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.867045 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3246ad-7dc3-4491-94ff-5daf3b286a26 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.867412 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3246ad-7dc3-4491-94ff-5daf3b286a26 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.869040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1094/2213] 10.222.0.86 () {54 vars in 921 bytes} [Wed Aug 24 10:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:04:09.899896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba998bc0-fa83-4258-ac9d-ae3d9b7ab7cd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.900302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba998bc0-fa83-4258-ac9d-ae3d9b7ab7cd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.900521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba998bc0-fa83-4258-ac9d-ae3d9b7ab7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.910637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:09.933096 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ba998bc0-fa83-4258-ac9d-ae3d9b7ab7cd None None] MFA Rules not processed for user `ef8d30e87c624d6180656d998afcebe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:09.937640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ba998bc0-fa83-4258-ac9d-ae3d9b7ab7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:09.990162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d97ba-723d-4b5f-91fa-6f6929d56885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:09.990874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d97ba-723d-4b5f-91fa-6f6929d56885 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.991384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d97ba-723d-4b5f-91fa-6f6929d56885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.991497 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d97ba-723d-4b5f-91fa-6f6929d56885 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.992184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d97ba-723d-4b5f-91fa-6f6929d56885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:09.994466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d97ba-723d-4b5f-91fa-6f6929d56885 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.052217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1120/2214] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:09 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:10.053321 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d97ba-723d-4b5f-91fa-6f6929d56885 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.055272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e30d97ba-723d-4b5f-91fa-6f6929d56885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:10.070534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d97ba-723d-4b5f-91fa-6f6929d56885 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.072365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1095/2215] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Aug 24 10:04:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:10.073319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba998bc0-fa83-4258-ac9d-ae3d9b7ab7cd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.077357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67333a7d-18a5-44ca-8820-4a579ad3762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.078478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67333a7d-18a5-44ca-8820-4a579ad3762a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.078800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67333a7d-18a5-44ca-8820-4a579ad3762a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.079032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67333a7d-18a5-44ca-8820-4a579ad3762a admin admin] PATH_INFO: `/v3/users/c4b8d531e382487a938546c009398b2e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.096104 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba998bc0-fa83-4258-ac9d-ae3d9b7ab7cd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.096198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67333a7d-18a5-44ca-8820-4a579ad3762a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4b8d531e382487a938546c009398b2e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.096198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67333a7d-18a5-44ca-8820-4a579ad3762a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.096352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df5264-c389-4248-9c89-29733829597f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.096352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5264-c389-4248-9c89-29733829597f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.096352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5264-c389-4248-9c89-29733829597f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.096352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5264-c389-4248-9c89-29733829597f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.096629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df5264-c389-4248-9c89-29733829597f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.096629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df5264-c389-4248-9c89-29733829597f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.108791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1096/2216] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:10.110057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df5264-c389-4248-9c89-29733829597f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.112956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-54df5264-c389-4248-9c89-29733829597f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:10.134067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df5264-c389-4248-9c89-29733829597f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.140850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531797e-c91c-4152-9d09-46f24757849f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.141504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531797e-c91c-4152-9d09-46f24757849f service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.141841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531797e-c91c-4152-9d09-46f24757849f service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.142241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531797e-c91c-4152-9d09-46f24757849f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.152143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531797e-c91c-4152-9d09-46f24757849f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.153946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531797e-c91c-4152-9d09-46f24757849f service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.159837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-67333a7d-18a5-44ca-8820-4a579ad3762a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4b8d531e382487a938546c009398b2e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.160601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-67333a7d-18a5-44ca-8820-4a579ad3762a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4b8d531e382487a938546c009398b2e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.161287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-67333a7d-18a5-44ca-8820-4a579ad3762a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4b8d531e382487a938546c009398b2e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.164428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1097/2217] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:10.166944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531797e-c91c-4152-9d09-46f24757849f service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.172353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-67333a7d-18a5-44ca-8820-4a579ad3762a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4b8d531e382487a938546c009398b2e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.179175 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531797e-c91c-4152-9d09-46f24757849f service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.183288 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-67333a7d-18a5-44ca-8820-4a579ad3762a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4b8d531e382487a938546c009398b2e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.186162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537f334-e6bb-4cc0-8ac9-49d1299b730d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.186786 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537f334-e6bb-4cc0-8ac9-49d1299b730d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.187118 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537f334-e6bb-4cc0-8ac9-49d1299b730d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.187324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537f334-e6bb-4cc0-8ac9-49d1299b730d admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.188430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537f334-e6bb-4cc0-8ac9-49d1299b730d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.190529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537f334-e6bb-4cc0-8ac9-49d1299b730d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.207593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1098/2218] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:10.209128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537f334-e6bb-4cc0-8ac9-49d1299b730d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.211178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2537f334-e6bb-4cc0-8ac9-49d1299b730d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:10.216430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1121/2219] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:10 2022] DELETE /identity/v3/users/c4b8d531e382487a938546c009398b2e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:10.225256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537f334-e6bb-4cc0-8ac9-49d1299b730d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.297822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e4b6b-9585-428e-97fc-73df905dada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.298765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e4b6b-9585-428e-97fc-73df905dada7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.299247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e4b6b-9585-428e-97fc-73df905dada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.299846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e4b6b-9585-428e-97fc-73df905dada7 admin admin] PATH_INFO: `/v3/projects/c887fdad15f34f59aaaa17536c52b3a7/users/61e2e7396999497782a4b9cfea65a02d/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.318962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e4b6b-9585-428e-97fc-73df905dada7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c887fdad15f34f59aaaa17536c52b3a7, user_id=61e2e7396999497782a4b9cfea65a02d, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.322530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e4b6b-9585-428e-97fc-73df905dada7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.362715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1099/2220] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:10 2022] PUT /identity/v3/projects/c887fdad15f34f59aaaa17536c52b3a7/users/61e2e7396999497782a4b9cfea65a02d/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:10.367209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc846e24-fab1-4b3d-b8e7-e30bcb7b2b48 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.367869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc846e24-fab1-4b3d-b8e7-e30bcb7b2b48 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.368476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc846e24-fab1-4b3d-b8e7-e30bcb7b2b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.390828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:10.410761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e4b6b-9585-428e-97fc-73df905dada7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.412201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-430e4b6b-9585-428e-97fc-73df905dada7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:10.433398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-bc846e24-fab1-4b3d-b8e7-e30bcb7b2b48 None None] MFA Rules not processed for user `61e2e7396999497782a4b9cfea65a02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:10.435765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bc846e24-fab1-4b3d-b8e7-e30bcb7b2b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:10.487505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:10.506216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e4b6b-9585-428e-97fc-73df905dada7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.511222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.511852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.512161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.512434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] PATH_INFO: `/v3/projects/8d455181e7e046f480437a8ab3620746` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.513581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d455181e7e046f480437a8ab3620746)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.515485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.520917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1122/2221] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:10.533275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc846e24-fab1-4b3d-b8e7-e30bcb7b2b48 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.534628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bc846e24-fab1-4b3d-b8e7-e30bcb7b2b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:10.567536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d455181e7e046f480437a8ab3620746', 'request_id': 'req-b55c54aa-9def-475e-ba15-291bca5a244b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.576290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d455181e7e046f480437a8ab3620746', 'request_id': 'req-b55c54aa-9def-475e-ba15-291bca5a244b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.583150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d455181e7e046f480437a8ab3620746', 'request_id': 'req-b55c54aa-9def-475e-ba15-291bca5a244b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.598176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] The token cache is being invalidate because project 8d455181e7e046f480437a8ab3620746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:10.598414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.599230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b55c54aa-9def-475e-ba15-291bca5a244b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.600978 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1100/2222] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:10 2022] DELETE /identity/v3/projects/8d455181e7e046f480437a8ab3620746 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:10.606093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fface7d9-f37e-45f7-82d3-68b5820305fc None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.606453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fface7d9-f37e-45f7-82d3-68b5820305fc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.607299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fface7d9-f37e-45f7-82d3-68b5820305fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.608781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc846e24-fab1-4b3d-b8e7-e30bcb7b2b48 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.628164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fface7d9-f37e-45f7-82d3-68b5820305fc None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:10.628886 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fface7d9-f37e-45f7-82d3-68b5820305fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:10.666626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc36a48-ce87-4eff-aef0-c75fbf21c25d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.667361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc36a48-ce87-4eff-aef0-c75fbf21c25d service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.667710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc36a48-ce87-4eff-aef0-c75fbf21c25d service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.668062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc36a48-ce87-4eff-aef0-c75fbf21c25d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.670535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8dc36a48-ce87-4eff-aef0-c75fbf21c25d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:10.721554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1101/2223] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:10.722941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce533c08-5e8a-4ccd-a3e7-028036794798 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.723413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce533c08-5e8a-4ccd-a3e7-028036794798 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.723695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce533c08-5e8a-4ccd-a3e7-028036794798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.749121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ce533c08-5e8a-4ccd-a3e7-028036794798 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:10.750327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ce533c08-5e8a-4ccd-a3e7-028036794798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:10.759965 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1102/2224] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:10.764611 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce533c08-5e8a-4ccd-a3e7-028036794798 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.776261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce533c08-5e8a-4ccd-a3e7-028036794798 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.781968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc36a48-ce87-4eff-aef0-c75fbf21c25d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.783512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc36a48-ce87-4eff-aef0-c75fbf21c25d service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.791854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496cda8-8b6a-40fe-a611-72b0adf0cde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.792519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496cda8-8b6a-40fe-a611-72b0adf0cde1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.792801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496cda8-8b6a-40fe-a611-72b0adf0cde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.793079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496cda8-8b6a-40fe-a611-72b0adf0cde1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.793743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496cda8-8b6a-40fe-a611-72b0adf0cde1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:10.794083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496cda8-8b6a-40fe-a611-72b0adf0cde1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.796550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496cda8-8b6a-40fe-a611-72b0adf0cde1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.797219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1123/2225] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:04:10.814371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1103/2226] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:10.817660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496cda8-8b6a-40fe-a611-72b0adf0cde1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.819423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc36a48-ce87-4eff-aef0-c75fbf21c25d service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.819912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5496cda8-8b6a-40fe-a611-72b0adf0cde1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:10.825155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:10.831302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc36a48-ce87-4eff-aef0-c75fbf21c25d service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.834098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496cda8-8b6a-40fe-a611-72b0adf0cde1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.835941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f180063-2fc3-4d9c-90c3-e6f56b7b5e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.836568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f180063-2fc3-4d9c-90c3-e6f56b7b5e5f admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.836853 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f180063-2fc3-4d9c-90c3-e6f56b7b5e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.837128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f180063-2fc3-4d9c-90c3-e6f56b7b5e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.837787 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f180063-2fc3-4d9c-90c3-e6f56b7b5e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.839224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e4a3e-0187-48ed-a67c-e63a3eedb9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.839838 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e4a3e-0187-48ed-a67c-e63a3eedb9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.839936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f180063-2fc3-4d9c-90c3-e6f56b7b5e5f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.840143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e4a3e-0187-48ed-a67c-e63a3eedb9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.840417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e4a3e-0187-48ed-a67c-e63a3eedb9fe admin admin] PATH_INFO: `/v3/users/61e2e7396999497782a4b9cfea65a02d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.842039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e4a3e-0187-48ed-a67c-e63a3eedb9fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=61e2e7396999497782a4b9cfea65a02d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.843955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e4a3e-0187-48ed-a67c-e63a3eedb9fe admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.868709 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1124/2227] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:10 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:10.873217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f180063-2fc3-4d9c-90c3-e6f56b7b5e5f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.899782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-756e4a3e-0187-48ed-a67c-e63a3eedb9fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61e2e7396999497782a4b9cfea65a02d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.900541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-756e4a3e-0187-48ed-a67c-e63a3eedb9fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61e2e7396999497782a4b9cfea65a02d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.901281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-756e4a3e-0187-48ed-a67c-e63a3eedb9fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61e2e7396999497782a4b9cfea65a02d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.906185 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:10.910623 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-756e4a3e-0187-48ed-a67c-e63a3eedb9fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61e2e7396999497782a4b9cfea65a02d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.921150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-756e4a3e-0187-48ed-a67c-e63a3eedb9fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61e2e7396999497782a4b9cfea65a02d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:10.940968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1104/2228] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:10 2022] DELETE /identity/v3/users/61e2e7396999497782a4b9cfea65a02d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:10.945479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0d5bb-8e05-4c0b-8852-74c208e4297b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.945765 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0d5bb-8e05-4c0b-8852-74c208e4297b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.946043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f180063-2fc3-4d9c-90c3-e6f56b7b5e5f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.946183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0d5bb-8e05-4c0b-8852-74c208e4297b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.966889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ebb0d5bb-8e05-4c0b-8852-74c208e4297b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:10.968147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ebb0d5bb-8e05-4c0b-8852-74c208e4297b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.010397 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3f17e-39f3-42a3-a339-5f496c7bfd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.011025 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3f17e-39f3-42a3-a339-5f496c7bfd9a admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.011328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3f17e-39f3-42a3-a339-5f496c7bfd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.011588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3f17e-39f3-42a3-a339-5f496c7bfd9a admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.012704 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3f17e-39f3-42a3-a339-5f496c7bfd9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.015851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3f17e-39f3-42a3-a339-5f496c7bfd9a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.042622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1105/2229] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:11.053615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1125/2230] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:10 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:11.053766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0d5bb-8e05-4c0b-8852-74c208e4297b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.053766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ebb0d5bb-8e05-4c0b-8852-74c208e4297b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.055305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3f17e-39f3-42a3-a339-5f496c7bfd9a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.057037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dad3f17e-39f3-42a3-a339-5f496c7bfd9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.069949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3f17e-39f3-42a3-a339-5f496c7bfd9a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.074628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f2db0-22e1-459c-900a-fb02ec967826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.075788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f2db0-22e1-459c-900a-fb02ec967826 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.076110 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f2db0-22e1-459c-900a-fb02ec967826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.076383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f2db0-22e1-459c-900a-fb02ec967826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.077032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f2db0-22e1-459c-900a-fb02ec967826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.078635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f2db0-22e1-459c-900a-fb02ec967826 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.089549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1126/2231] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:11.094057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f2db0-22e1-459c-900a-fb02ec967826 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.103927 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f2db0-22e1-459c-900a-fb02ec967826 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.108646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85488ed4-2e1d-4bd1-8671-94ced95a437c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.109483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85488ed4-2e1d-4bd1-8671-94ced95a437c admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.109759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85488ed4-2e1d-4bd1-8671-94ced95a437c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.110027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85488ed4-2e1d-4bd1-8671-94ced95a437c admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.110658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85488ed4-2e1d-4bd1-8671-94ced95a437c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.112259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85488ed4-2e1d-4bd1-8671-94ced95a437c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.118270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0d5bb-8e05-4c0b-8852-74c208e4297b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.122312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae6f62-0b2a-4ca2-aaaa-adde20702131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.122962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1127/2232] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:11.124105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6f62-0b2a-4ca2-aaaa-adde20702131 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.124418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6f62-0b2a-4ca2-aaaa-adde20702131 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.124695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6f62-0b2a-4ca2-aaaa-adde20702131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.126800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85488ed4-2e1d-4bd1-8671-94ced95a437c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.134341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae6f62-0b2a-4ca2-aaaa-adde20702131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.135771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae6f62-0b2a-4ca2-aaaa-adde20702131 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.136242 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85488ed4-2e1d-4bd1-8671-94ced95a437c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.141244 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87766fe0-4512-43de-8e9c-ea688bc7dc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.141861 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87766fe0-4512-43de-8e9c-ea688bc7dc44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.142134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87766fe0-4512-43de-8e9c-ea688bc7dc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.142405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87766fe0-4512-43de-8e9c-ea688bc7dc44 admin admin] PATH_INFO: `/v3/projects/e055ba4967db42108c069ebc3843dcf6/users/1de97cce55e944dbb62a920ea11734f5/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.146016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1106/2233] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:11.157707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87766fe0-4512-43de-8e9c-ea688bc7dc44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e055ba4967db42108c069ebc3843dcf6, user_id=1de97cce55e944dbb62a920ea11734f5, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.160436 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87766fe0-4512-43de-8e9c-ea688bc7dc44 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.174202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1128/2234] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:11 2022] PUT /identity/v3/projects/e055ba4967db42108c069ebc3843dcf6/users/1de97cce55e944dbb62a920ea11734f5/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:11.179219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffabbb5-8f5e-43e3-91c2-85fa6e4a768c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.179709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffabbb5-8f5e-43e3-91c2-85fa6e4a768c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.179790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffabbb5-8f5e-43e3-91c2-85fa6e4a768c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.200138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9ffabbb5-8f5e-43e3-91c2-85fa6e4a768c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:11.201877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9ffabbb5-8f5e-43e3-91c2-85fa6e4a768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.265992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1107/2235] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:11.275745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87766fe0-4512-43de-8e9c-ea688bc7dc44 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.277615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-87766fe0-4512-43de-8e9c-ea688bc7dc44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.318173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:11.344913 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87766fe0-4512-43de-8e9c-ea688bc7dc44 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.348804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b3da7-94a2-4ee9-9da0-cad97e74150f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.349405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3da7-94a2-4ee9-9da0-cad97e74150f service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.349686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3da7-94a2-4ee9-9da0-cad97e74150f service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.349960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3da7-94a2-4ee9-9da0-cad97e74150f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.359540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b3da7-94a2-4ee9-9da0-cad97e74150f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.361004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b3da7-94a2-4ee9-9da0-cad97e74150f service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.370821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1129/2236] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:11.477376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffabbb5-8f5e-43e3-91c2-85fa6e4a768c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.478936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9ffabbb5-8f5e-43e3-91c2-85fa6e4a768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.492353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffabbb5-8f5e-43e3-91c2-85fa6e4a768c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.512118 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.513460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.513690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.513905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] PATH_INFO: `/v3/projects/c887fdad15f34f59aaaa17536c52b3a7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.515130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c887fdad15f34f59aaaa17536c52b3a7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.517195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.525918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:11.568062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c887fdad15f34f59aaaa17536c52b3a7', 'request_id': 'req-1a324127-b41c-415f-8e6c-278e0bf7266f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:11.576726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c887fdad15f34f59aaaa17536c52b3a7', 'request_id': 'req-1a324127-b41c-415f-8e6c-278e0bf7266f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:11.582641 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c887fdad15f34f59aaaa17536c52b3a7', 'request_id': 'req-1a324127-b41c-415f-8e6c-278e0bf7266f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:11.599241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] The token cache is being invalidate because project c887fdad15f34f59aaaa17536c52b3a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:11.599540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:11.600432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1a324127-b41c-415f-8e6c-278e0bf7266f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:11.602246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1108/2237] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:11 2022] DELETE /identity/v3/projects/c887fdad15f34f59aaaa17536c52b3a7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:11.608234 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66b45-8b16-487d-893a-315a21de3c63 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.608736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66b45-8b16-487d-893a-315a21de3c63 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.608915 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66b45-8b16-487d-893a-315a21de3c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.630287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2ac66b45-8b16-487d-893a-315a21de3c63 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:11.631660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2ac66b45-8b16-487d-893a-315a21de3c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.697562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1130/2238] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:11.705568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646a82-398a-496b-a538-1bb0013c8430 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.706215 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646a82-398a-496b-a538-1bb0013c8430 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.706501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646a82-398a-496b-a538-1bb0013c8430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.729718 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0e646a82-398a-496b-a538-1bb0013c8430 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:11.731396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0e646a82-398a-496b-a538-1bb0013c8430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.744431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1109/2239] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:11.746737 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac66b45-8b16-487d-893a-315a21de3c63 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.757663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac66b45-8b16-487d-893a-315a21de3c63 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.764573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5917123-d720-42d2-9428-f06638d367f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.765198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5917123-d720-42d2-9428-f06638d367f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.765480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5917123-d720-42d2-9428-f06638d367f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.765757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5917123-d720-42d2-9428-f06638d367f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.766425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5917123-d720-42d2-9428-f06638d367f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:11.766729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5917123-d720-42d2-9428-f06638d367f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.771646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5917123-d720-42d2-9428-f06638d367f1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.827736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1131/2240] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:11.832323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e646a82-398a-496b-a538-1bb0013c8430 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.842136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e646a82-398a-496b-a538-1bb0013c8430 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.853098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ece9da-e963-4982-8868-07f79c263a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.853808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ece9da-e963-4982-8868-07f79c263a2a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.854662 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ece9da-e963-4982-8868-07f79c263a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.854662 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ece9da-e963-4982-8868-07f79c263a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.855440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ece9da-e963-4982-8868-07f79c263a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.859065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ece9da-e963-4982-8868-07f79c263a2a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.896832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1110/2241] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:11 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:11.898070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5917123-d720-42d2-9428-f06638d367f1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.911062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5917123-d720-42d2-9428-f06638d367f1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.998737 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3417116-776d-48fa-b939-a6cff941f636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.999358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3417116-776d-48fa-b939-a6cff941f636 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.999665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3417116-776d-48fa-b939-a6cff941f636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.999945 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3417116-776d-48fa-b939-a6cff941f636 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:12.000571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3417116-776d-48fa-b939-a6cff941f636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:12.003225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3417116-776d-48fa-b939-a6cff941f636 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:12.034147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1132/2242] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:11 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:12.036915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ece9da-e963-4982-8868-07f79c263a2a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:12.068810 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ece9da-e963-4982-8868-07f79c263a2a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:12.100382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545453b-b921-4e42-a5ec-c7c7f13f5b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:12.101352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545453b-b921-4e42-a5ec-c7c7f13f5b08 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:12.102212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545453b-b921-4e42-a5ec-c7c7f13f5b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:12.102723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545453b-b921-4e42-a5ec-c7c7f13f5b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:12.104213 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545453b-b921-4e42-a5ec-c7c7f13f5b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:12.109794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545453b-b921-4e42-a5ec-c7c7f13f5b08 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:12.129817 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1111/2243] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:12.133981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3417116-776d-48fa-b939-a6cff941f636 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:12.158389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3417116-776d-48fa-b939-a6cff941f636 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:12.163610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162222b-adbe-4ed7-8be6-6e73744af379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:12.164472 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162222b-adbe-4ed7-8be6-6e73744af379 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:12.164798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162222b-adbe-4ed7-8be6-6e73744af379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:12.165119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162222b-adbe-4ed7-8be6-6e73744af379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:12.166031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162222b-adbe-4ed7-8be6-6e73744af379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:12.168398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162222b-adbe-4ed7-8be6-6e73744af379 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:12.184083 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1133/2244] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:12.185705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545453b-b921-4e42-a5ec-c7c7f13f5b08 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:12.199135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545453b-b921-4e42-a5ec-c7c7f13f5b08 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:12.204607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d12d1a-ef2b-4b74-986d-74e4d4fc16cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:12.205323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d12d1a-ef2b-4b74-986d-74e4d4fc16cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:12.205608 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d12d1a-ef2b-4b74-986d-74e4d4fc16cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:12.205884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d12d1a-ef2b-4b74-986d-74e4d4fc16cc admin admin] PATH_INFO: `/v3/projects/302c5ab2e660436e8a0af25c12ceee20/users/1c665588c50d47dbaf1f8fde082ee73a/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:12.223252 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d12d1a-ef2b-4b74-986d-74e4d4fc16cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=302c5ab2e660436e8a0af25c12ceee20, user_id=1c665588c50d47dbaf1f8fde082ee73a, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:12.227013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d12d1a-ef2b-4b74-986d-74e4d4fc16cc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:12.246740 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1112/2245] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:12 2022] PUT /identity/v3/projects/302c5ab2e660436e8a0af25c12ceee20/users/1c665588c50d47dbaf1f8fde082ee73a/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:12.248462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d8409-244d-4603-a714-d7adc73b8ecb None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:12.248753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d8409-244d-4603-a714-d7adc73b8ecb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:12.249032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d8409-244d-4603-a714-d7adc73b8ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:12.268497 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e00d8409-244d-4603-a714-d7adc73b8ecb None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:12.269735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e00d8409-244d-4603-a714-d7adc73b8ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:12.327193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:12.353160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1134/2246] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:12.364735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d12d1a-ef2b-4b74-986d-74e4d4fc16cc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:12.366957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-50d12d1a-ef2b-4b74-986d-74e4d4fc16cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:12.443870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d12d1a-ef2b-4b74-986d-74e4d4fc16cc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:12.448241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881bb53-14fa-425e-a422-fc002d2efa58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:12.449092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881bb53-14fa-425e-a422-fc002d2efa58 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:12.449607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881bb53-14fa-425e-a422-fc002d2efa58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:12.450729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881bb53-14fa-425e-a422-fc002d2efa58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:12.464127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881bb53-14fa-425e-a422-fc002d2efa58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:12.466565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881bb53-14fa-425e-a422-fc002d2efa58 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:12.476588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1113/2247] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:14.605365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d8409-244d-4603-a714-d7adc73b8ecb None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:14.607501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e00d8409-244d-4603-a714-d7adc73b8ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:14.630501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d8409-244d-4603-a714-d7adc73b8ecb None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:14.643943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:14.644639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:14.644926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:14.645212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] PATH_INFO: `/v3/users/558feb74daa446e6b3af89bed28622a9` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:14.649484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] RBAC: Authorizing `identity:delete_user(user_id=558feb74daa446e6b3af89bed28622a9)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:14.651514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:14.754009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '558feb74daa446e6b3af89bed28622a9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:14.758498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '558feb74daa446e6b3af89bed28622a9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:14.759223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '558feb74daa446e6b3af89bed28622a9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:14.781746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '558feb74daa446e6b3af89bed28622a9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:14.792536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '558feb74daa446e6b3af89bed28622a9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:14.831362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af4884-e08b-480e-a558-a5878cb637d5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:14.831872 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af4884-e08b-480e-a558-a5878cb637d5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:14.831972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af4884-e08b-480e-a558-a5878cb637d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:14.834890 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1135/2248] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:14 2022] DELETE /identity/v3/users/558feb74daa446e6b3af89bed28622a9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:14.923669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-67af4884-e08b-480e-a558-a5878cb637d5 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:14.926902 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-67af4884-e08b-480e-a558-a5878cb637d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:15.050010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:15.157160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:15.159568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:15.169456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1114/2249] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:15.179793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af4884-e08b-480e-a558-a5878cb637d5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:15.181336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-67af4884-e08b-480e-a558-a5878cb637d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:15.299212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af4884-e08b-480e-a558-a5878cb637d5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:15.311399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1219d8-a2d7-436c-97b2-441bd908907f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:15.312725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1219d8-a2d7-436c-97b2-441bd908907f service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:15.313019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1219d8-a2d7-436c-97b2-441bd908907f service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:15.313349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1219d8-a2d7-436c-97b2-441bd908907f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:15.313531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8d668-63fa-4322-bc7a-d393d3b5334c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:15.322496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34d9cc-312f-42ac-a378-82a03eaefabb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:15.324585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34d9cc-312f-42ac-a378-82a03eaefabb service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:15.325803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34d9cc-312f-42ac-a378-82a03eaefabb service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:15.328366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34d9cc-312f-42ac-a378-82a03eaefabb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:15.330940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1219d8-a2d7-436c-97b2-441bd908907f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:15.333030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1219d8-a2d7-436c-97b2-441bd908907f service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:15.347281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1115/2250] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:15.357452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0b34d9cc-312f-42ac-a378-82a03eaefabb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:15.581653 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34d9cc-312f-42ac-a378-82a03eaefabb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:15.583671 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34d9cc-312f-42ac-a378-82a03eaefabb service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:15.620748 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1136/2251] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:04:16.521331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1219d8-a2d7-436c-97b2-441bd908907f service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:16.523039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fa1219d8-a2d7-436c-97b2-441bd908907f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:16.547279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1219d8-a2d7-436c-97b2-441bd908907f service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:16.555825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce47e15-f034-4165-9a14-4d4d4e854bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:16.556900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce47e15-f034-4165-9a14-4d4d4e854bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.557414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce47e15-f034-4165-9a14-4d4d4e854bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.558911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce47e15-f034-4165-9a14-4d4d4e854bfb admin admin] PATH_INFO: `/v3/users/0e39cf9dbd264e71bacf8a26aa21fb5c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.562017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce47e15-f034-4165-9a14-4d4d4e854bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e39cf9dbd264e71bacf8a26aa21fb5c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:16.565368 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce47e15-f034-4165-9a14-4d4d4e854bfb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:16.619284 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34d9cc-312f-42ac-a378-82a03eaefabb service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:16.621361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0b34d9cc-312f-42ac-a378-82a03eaefabb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:16.645352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:16.647506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dce47e15-f034-4165-9a14-4d4d4e854bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e39cf9dbd264e71bacf8a26aa21fb5c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.648991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dce47e15-f034-4165-9a14-4d4d4e854bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e39cf9dbd264e71bacf8a26aa21fb5c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.650628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dce47e15-f034-4165-9a14-4d4d4e854bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e39cf9dbd264e71bacf8a26aa21fb5c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.662807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34d9cc-312f-42ac-a378-82a03eaefabb service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:16.672277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dce47e15-f034-4165-9a14-4d4d4e854bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e39cf9dbd264e71bacf8a26aa21fb5c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.683091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:16.685207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.687586 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.689150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] PATH_INFO: `/v3/projects/ca73c0a36d394b0fa47c5b7e9b09ef59` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.692834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca73c0a36d394b0fa47c5b7e9b09ef59)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:16.695584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dce47e15-f034-4165-9a14-4d4d4e854bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e39cf9dbd264e71bacf8a26aa21fb5c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.697850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:16.737586 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1116/2252] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:16 2022] DELETE /identity/v3/users/0e39cf9dbd264e71bacf8a26aa21fb5c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:16.742719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862f00a-e871-42cd-b093-b92289606d67 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.743031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862f00a-e871-42cd-b093-b92289606d67 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.743201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862f00a-e871-42cd-b093-b92289606d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.814914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4862f00a-e871-42cd-b093-b92289606d67 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:16.823348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4862f00a-e871-42cd-b093-b92289606d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:16.845903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca73c0a36d394b0fa47c5b7e9b09ef59', 'request_id': 'req-182ca522-ebcd-45b5-9302-03d3e909d15a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.857740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca73c0a36d394b0fa47c5b7e9b09ef59', 'request_id': 'req-182ca522-ebcd-45b5-9302-03d3e909d15a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.876160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca73c0a36d394b0fa47c5b7e9b09ef59', 'request_id': 'req-182ca522-ebcd-45b5-9302-03d3e909d15a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.917974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] The token cache is being invalidate because project ca73c0a36d394b0fa47c5b7e9b09ef59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:16.919166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.920503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.923547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1137/2253] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:16 2022] DELETE /identity/v3/projects/ca73c0a36d394b0fa47c5b7e9b09ef59 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:16.928621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:16.938495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:16.947341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:16.996505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1117/2254] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:17.025281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862f00a-e871-42cd-b093-b92289606d67 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:17.026989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4862f00a-e871-42cd-b093-b92289606d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:17.120282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ca522-ebcd-45b5-9302-03d3e909d15a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:17.146722 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:17.147380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:17.147700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:17.148204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] PATH_INFO: `/v3/users/c656be83dbe44671a1d30004d4521332` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:17.149808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c656be83dbe44671a1d30004d4521332)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:17.152297 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:17.202689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862f00a-e871-42cd-b093-b92289606d67 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:17.224854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900a04a-c3d4-479e-9666-e12a1d5eec9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:17.225518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900a04a-c3d4-479e-9666-e12a1d5eec9e service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:17.225806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900a04a-c3d4-479e-9666-e12a1d5eec9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:17.226140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900a04a-c3d4-479e-9666-e12a1d5eec9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:17.244502 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900a04a-c3d4-479e-9666-e12a1d5eec9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:17.245966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900a04a-c3d4-479e-9666-e12a1d5eec9e service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:17.262698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c656be83dbe44671a1d30004d4521332'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:17.263564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c656be83dbe44671a1d30004d4521332'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:17.264339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c656be83dbe44671a1d30004d4521332'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:17.277615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c656be83dbe44671a1d30004d4521332'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:17.286733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1118/2255] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:17.300609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c656be83dbe44671a1d30004d4521332'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:17.343179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1138/2256] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:16 2022] DELETE /identity/v3/users/c656be83dbe44671a1d30004d4521332 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:18.423617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ca09c-6011-4232-91b1-4821c079acc3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:18.424257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ca09c-6011-4232-91b1-4821c079acc3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:18.424257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ca09c-6011-4232-91b1-4821c079acc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:18.493984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:18.495714 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:18.498694 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-525ca09c-6011-4232-91b1-4821c079acc3 None None] MFA Rules not processed for user `1de97cce55e944dbb62a920ea11734f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:18.501393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-525ca09c-6011-4232-91b1-4821c079acc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:18.519268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:18.693669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1119/2257] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:18.694158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe20a1c-c877-4d9e-b8f7-2e9437f135f8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:18.708537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:18.710732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:18.710732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:18.710732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] PATH_INFO: `/v3/projects/92ba51632c0646f5aa59afa413da65ec` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:18.711337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ca09c-6011-4232-91b1-4821c079acc3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:18.715257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-525ca09c-6011-4232-91b1-4821c079acc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:18.718354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ba51632c0646f5aa59afa413da65ec)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:18.748222 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:18.885580 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ba51632c0646f5aa59afa413da65ec', 'request_id': 'req-501b489f-8b7f-4531-b040-3117c487e89a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:18.906611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ba51632c0646f5aa59afa413da65ec', 'request_id': 'req-501b489f-8b7f-4531-b040-3117c487e89a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:18.932385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ba51632c0646f5aa59afa413da65ec', 'request_id': 'req-501b489f-8b7f-4531-b040-3117c487e89a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:18.960677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] The token cache is being invalidate because project 92ba51632c0646f5aa59afa413da65ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:18.961030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:18.962368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:18.967748 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1139/2258] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:18 2022] DELETE /identity/v3/projects/92ba51632c0646f5aa59afa413da65ec => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:18.969638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:18.972784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:19.005207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ca09c-6011-4232-91b1-4821c079acc3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:19.075598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:19.186731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b489f-8b7f-4531-b040-3117c487e89a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:19.213715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccc291-3a71-47dd-ac29-660c774ba5e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:19.213851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:19.214470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccc291-3a71-47dd-ac29-660c774ba5e5 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:19.214601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:19.214774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:19.214879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccc291-3a71-47dd-ac29-660c774ba5e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:19.215068 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] PATH_INFO: `/v3/projects/f9087a2f9b9347809a53a2c3896cf8d0` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:19.215278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccc291-3a71-47dd-ac29-660c774ba5e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:19.218643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-beccc291-3a71-47dd-ac29-660c774ba5e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:19.220856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9087a2f9b9347809a53a2c3896cf8d0)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:19.222829 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:19.345099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:19.347248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccc291-3a71-47dd-ac29-660c774ba5e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:19.353054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccc291-3a71-47dd-ac29-660c774ba5e5 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:19.359125 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9087a2f9b9347809a53a2c3896cf8d0', 'request_id': 'req-272cb7af-6b59-4055-b3fa-ed8707b33727'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.367584 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9087a2f9b9347809a53a2c3896cf8d0', 'request_id': 'req-272cb7af-6b59-4055-b3fa-ed8707b33727'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.375409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9087a2f9b9347809a53a2c3896cf8d0', 'request_id': 'req-272cb7af-6b59-4055-b3fa-ed8707b33727'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.376072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1120/2259] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:19.377415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccc291-3a71-47dd-ac29-660c774ba5e5 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:19.380565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-beccc291-3a71-47dd-ac29-660c774ba5e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:19.391732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] The token cache is being invalidate because project f9087a2f9b9347809a53a2c3896cf8d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:19.391957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.392891 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.397485 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1140/2260] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:18 2022] DELETE /identity/v3/projects/f9087a2f9b9347809a53a2c3896cf8d0 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:19.401172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:19.406641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:19.416770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccc291-3a71-47dd-ac29-660c774ba5e5 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:19.515233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cb7af-6b59-4055-b3fa-ed8707b33727 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:19.522069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:19.522785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:19.523072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:19.523339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] PATH_INFO: `/v3/users/a481a88bef9e43ed85c81f7837c260a2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:19.524564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a481a88bef9e43ed85c81f7837c260a2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:19.526487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:19.527385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:19.527873 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:19.528199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:19.528609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] PATH_INFO: `/v3/users/299e8544a4694b089508927f51c07307` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:19.529815 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=299e8544a4694b089508927f51c07307)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:19.531884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:19.632748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299e8544a4694b089508927f51c07307'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.638632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299e8544a4694b089508927f51c07307'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.639555 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299e8544a4694b089508927f51c07307'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.646974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a481a88bef9e43ed85c81f7837c260a2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.647787 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a481a88bef9e43ed85c81f7837c260a2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.648632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a481a88bef9e43ed85c81f7837c260a2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.655343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299e8544a4694b089508927f51c07307'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.667214 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a481a88bef9e43ed85c81f7837c260a2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.675556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:19.678195 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299e8544a4694b089508927f51c07307'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.682322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a481a88bef9e43ed85c81f7837c260a2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.705784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1121/2261] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:19 2022] DELETE /identity/v3/users/299e8544a4694b089508927f51c07307 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:19.725020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1141/2262] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:19 2022] DELETE /identity/v3/users/a481a88bef9e43ed85c81f7837c260a2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:20.765940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:20.769144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:20.773970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:20.776649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:20.907603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:20.941667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a602cf-17fe-4e8f-a3a3-dcfaff91e333 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:20.949514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1c2e3-85f8-4f29-bd60-6351149532b3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:20.958714 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:20.959452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:20.959739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:20.960016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] PATH_INFO: `/v3/projects/e909f1c8d76e43569de975fec698bb93` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:20.961287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:20.961555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e909f1c8d76e43569de975fec698bb93)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:20.961984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:20.963419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:20.963714 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] PATH_INFO: `/v3/projects/55c41585636c4629a92859a2a23c3432` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:20.965039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c41585636c4629a92859a2a23c3432)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:20.967713 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:20.972788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.174323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e909f1c8d76e43569de975fec698bb93', 'request_id': 'req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.179773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c41585636c4629a92859a2a23c3432', 'request_id': 'req-9bddf6e2-942c-4fb6-abce-031d7401bbe1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.190032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c41585636c4629a92859a2a23c3432', 'request_id': 'req-9bddf6e2-942c-4fb6-abce-031d7401bbe1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.191155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e909f1c8d76e43569de975fec698bb93', 'request_id': 'req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.208882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e909f1c8d76e43569de975fec698bb93', 'request_id': 'req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.232733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c41585636c4629a92859a2a23c3432', 'request_id': 'req-9bddf6e2-942c-4fb6-abce-031d7401bbe1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.242543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] The token cache is being invalidate because project e909f1c8d76e43569de975fec698bb93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:21.243654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.246546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e7e8213a-63d4-4f33-a34f-a9aaf55ccdfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.248799 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1142/2263] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:20 2022] DELETE /identity/v3/projects/e909f1c8d76e43569de975fec698bb93 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:21.253880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07f7e3-631b-48b4-9f70-b55db6f628d7 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.254170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07f7e3-631b-48b4-9f70-b55db6f628d7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.254498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07f7e3-631b-48b4-9f70-b55db6f628d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.286833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] The token cache is being invalidate because project 55c41585636c4629a92859a2a23c3432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:21.290428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.294021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9bddf6e2-942c-4fb6-abce-031d7401bbe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.296174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1122/2264] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:20 2022] DELETE /identity/v3/projects/55c41585636c4629a92859a2a23c3432 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:21.302003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407d7fe-b0ca-4bbd-8fb9-0201a1408fa6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.302003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407d7fe-b0ca-4bbd-8fb9-0201a1408fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.302003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407d7fe-b0ca-4bbd-8fb9-0201a1408fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.314589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-bd07f7e3-631b-48b4-9f70-b55db6f628d7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:21.319160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bd07f7e3-631b-48b4-9f70-b55db6f628d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:21.350216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3407d7fe-b0ca-4bbd-8fb9-0201a1408fa6 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:21.353296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3407d7fe-b0ca-4bbd-8fb9-0201a1408fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:21.484165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1123/2265] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:21.489525 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e25c9-5f1c-4f65-a661-d9602cd656a9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.490582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e25c9-5f1c-4f65-a661-d9602cd656a9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.491038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e25c9-5f1c-4f65-a661-d9602cd656a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.527761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bf8e25c9-5f1c-4f65-a661-d9602cd656a9 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:21.530330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bf8e25c9-5f1c-4f65-a661-d9602cd656a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:21.537448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1143/2266] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:21.538715 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07f7e3-631b-48b4-9f70-b55db6f628d7 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.541443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1124/2267] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:21.548427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e25c9-5f1c-4f65-a661-d9602cd656a9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.548620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bd07f7e3-631b-48b4-9f70-b55db6f628d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:21.557747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:21.566372 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e25c9-5f1c-4f65-a661-d9602cd656a9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.573912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4374381-144e-45b7-8b26-4ce4a293665e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.575410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4374381-144e-45b7-8b26-4ce4a293665e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.576036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4374381-144e-45b7-8b26-4ce4a293665e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.576684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4374381-144e-45b7-8b26-4ce4a293665e admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.577630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4374381-144e-45b7-8b26-4ce4a293665e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:21.578132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4374381-144e-45b7-8b26-4ce4a293665e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.580066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4374381-144e-45b7-8b26-4ce4a293665e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.590301 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07f7e3-631b-48b4-9f70-b55db6f628d7 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.607145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.616869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.617677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.618649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] PATH_INFO: `/v3/users/ebeca2571bac4b779e57a49eb1de5a60` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.619410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1125/2268] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:21.621389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebeca2571bac4b779e57a49eb1de5a60)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.624084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.627509 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51ea75-7d86-4587-a51b-31a05543d3b5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.627509 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51ea75-7d86-4587-a51b-31a05543d3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.627762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51ea75-7d86-4587-a51b-31a05543d3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.661805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-eb51ea75-7d86-4587-a51b-31a05543d3b5 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:21.663048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eb51ea75-7d86-4587-a51b-31a05543d3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:21.678436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1126/2269] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:21.680308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99778d23-3cb1-4667-bd5d-87e504c43aeb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.680764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99778d23-3cb1-4667-bd5d-87e504c43aeb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.681168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99778d23-3cb1-4667-bd5d-87e504c43aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.710340 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebeca2571bac4b779e57a49eb1de5a60'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.711532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebeca2571bac4b779e57a49eb1de5a60'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.712367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebeca2571bac4b779e57a49eb1de5a60'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.725801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-99778d23-3cb1-4667-bd5d-87e504c43aeb None None] MFA Rules not processed for user `1c665588c50d47dbaf1f8fde082ee73a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:21.726014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebeca2571bac4b779e57a49eb1de5a60'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.733714 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-99778d23-3cb1-4667-bd5d-87e504c43aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:21.744718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebeca2571bac4b779e57a49eb1de5a60'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:21.756451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:21.796758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1144/2270] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:21 2022] DELETE /identity/v3/users/ebeca2571bac4b779e57a49eb1de5a60 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:21.799023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.801410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:21.906991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1127/2271] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:21.909949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99778d23-3cb1-4667-bd5d-87e504c43aeb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.911763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-99778d23-3cb1-4667-bd5d-87e504c43aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:21.914609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2558b2f-759e-43c5-b96e-6ac8d58b7919 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.923952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2adfff-755d-4403-a3ee-59b7ebafea84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.924881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2adfff-755d-4403-a3ee-59b7ebafea84 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.928191 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2adfff-755d-4403-a3ee-59b7ebafea84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.928568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2adfff-755d-4403-a3ee-59b7ebafea84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.930177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2adfff-755d-4403-a3ee-59b7ebafea84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.934599 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2adfff-755d-4403-a3ee-59b7ebafea84 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.935075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99778d23-3cb1-4667-bd5d-87e504c43aeb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.941044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774a121-257d-4f09-a38e-1511f8acf1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.942055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774a121-257d-4f09-a38e-1511f8acf1cb admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.942486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774a121-257d-4f09-a38e-1511f8acf1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.942908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774a121-257d-4f09-a38e-1511f8acf1cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.943896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774a121-257d-4f09-a38e-1511f8acf1cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:21.945512 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774a121-257d-4f09-a38e-1511f8acf1cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.947333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774a121-257d-4f09-a38e-1511f8acf1cb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.967782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1128/2272] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:21.969520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2a468-de6f-4774-87de-0298f740ab83 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.970007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2a468-de6f-4774-87de-0298f740ab83 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.970296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2a468-de6f-4774-87de-0298f740ab83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.987972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1145/2273] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:21.987972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2adfff-755d-4403-a3ee-59b7ebafea84 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.990403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dd2adfff-755d-4403-a3ee-59b7ebafea84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:22.009676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-dfb2a468-de6f-4774-87de-0298f740ab83 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:22.011621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dfb2a468-de6f-4774-87de-0298f740ab83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:22.142285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2adfff-755d-4403-a3ee-59b7ebafea84 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.147954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858c631-23d7-424a-8af5-8fdeb12f6bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.148871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858c631-23d7-424a-8af5-8fdeb12f6bef admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.149155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858c631-23d7-424a-8af5-8fdeb12f6bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.149539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858c631-23d7-424a-8af5-8fdeb12f6bef admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.150569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858c631-23d7-424a-8af5-8fdeb12f6bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.152745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858c631-23d7-424a-8af5-8fdeb12f6bef admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.166171 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1129/2274] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:22.167277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2a468-de6f-4774-87de-0298f740ab83 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.194949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2a468-de6f-4774-87de-0298f740ab83 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.202425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1146/2275] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/projects => generated 371 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:22.203859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858c631-23d7-424a-8af5-8fdeb12f6bef admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.207425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d126718-56fc-438f-8732-20ee518d10df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.208217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d126718-56fc-438f-8732-20ee518d10df admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.208601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d126718-56fc-438f-8732-20ee518d10df admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.208979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d126718-56fc-438f-8732-20ee518d10df admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.209716 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d126718-56fc-438f-8732-20ee518d10df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.211750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d126718-56fc-438f-8732-20ee518d10df admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.247085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1130/2276] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:22 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:22.248286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d126718-56fc-438f-8732-20ee518d10df admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.250159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8d126718-56fc-438f-8732-20ee518d10df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:22.282817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:22.288164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858c631-23d7-424a-8af5-8fdeb12f6bef admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.322467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:22.343828 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d126718-56fc-438f-8732-20ee518d10df admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.348008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8db25-b045-4b5e-8b5b-d763a995610c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.348895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8db25-b045-4b5e-8b5b-d763a995610c service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.349732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8db25-b045-4b5e-8b5b-d763a995610c service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.349732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8db25-b045-4b5e-8b5b-d763a995610c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.392780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565913e6-eef3-4f70-8f1f-5d6f439530a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.393484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565913e6-eef3-4f70-8f1f-5d6f439530a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.393767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565913e6-eef3-4f70-8f1f-5d6f439530a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.394038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565913e6-eef3-4f70-8f1f-5d6f439530a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.394803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565913e6-eef3-4f70-8f1f-5d6f439530a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.396785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565913e6-eef3-4f70-8f1f-5d6f439530a5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.432140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1147/2277] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:22 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:22.432140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565913e6-eef3-4f70-8f1f-5d6f439530a5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.436192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8db25-b045-4b5e-8b5b-d763a995610c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.438172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8db25-b045-4b5e-8b5b-d763a995610c service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.443932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565913e6-eef3-4f70-8f1f-5d6f439530a5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.450133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328c29a-90e7-4c19-8bfa-e9aceef8fad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.450948 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328c29a-90e7-4c19-8bfa-e9aceef8fad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.451304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328c29a-90e7-4c19-8bfa-e9aceef8fad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.451652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328c29a-90e7-4c19-8bfa-e9aceef8fad2 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.452399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328c29a-90e7-4c19-8bfa-e9aceef8fad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.454534 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328c29a-90e7-4c19-8bfa-e9aceef8fad2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.455632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1131/2278] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:22.460381 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8db25-b045-4b5e-8b5b-d763a995610c service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.475902 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8db25-b045-4b5e-8b5b-d763a995610c service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.481703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a2d4a-763f-429e-8c49-86b810df0ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.482313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a2d4a-763f-429e-8c49-86b810df0ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.482605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a2d4a-763f-429e-8c49-86b810df0ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.482861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a2d4a-763f-429e-8c49-86b810df0ff6 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.483491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a2d4a-763f-429e-8c49-86b810df0ff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.485771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a2d4a-763f-429e-8c49-86b810df0ff6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.493179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1148/2279] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:22 2022] POST /identity/v3/users => generated 292 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:22.496989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328c29a-90e7-4c19-8bfa-e9aceef8fad2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.508842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328c29a-90e7-4c19-8bfa-e9aceef8fad2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.514065 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66920d1e-11fe-4aed-9045-7a07cf623dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.514685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66920d1e-11fe-4aed-9045-7a07cf623dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.514972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66920d1e-11fe-4aed-9045-7a07cf623dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.515321 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66920d1e-11fe-4aed-9045-7a07cf623dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.515981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66920d1e-11fe-4aed-9045-7a07cf623dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.517664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66920d1e-11fe-4aed-9045-7a07cf623dc4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.525158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1132/2280] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:22 2022] POST /identity/v3/users => generated 284 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:22.525158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a2d4a-763f-429e-8c49-86b810df0ff6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.545594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a2d4a-763f-429e-8c49-86b810df0ff6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.545773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1149/2281] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:22.547620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66920d1e-11fe-4aed-9045-7a07cf623dc4 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.557755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb31a4a-c5ad-459e-b29a-e80faecafbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.558483 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb31a4a-c5ad-459e-b29a-e80faecafbc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.558738 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb31a4a-c5ad-459e-b29a-e80faecafbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.559113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb31a4a-c5ad-459e-b29a-e80faecafbc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.559794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb31a4a-c5ad-459e-b29a-e80faecafbc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.563229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb31a4a-c5ad-459e-b29a-e80faecafbc5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.572108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66920d1e-11fe-4aed-9045-7a07cf623dc4 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.579281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d47b0-1f32-4d37-a68f-61591097a5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.580312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d47b0-1f32-4d37-a68f-61591097a5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.580846 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d47b0-1f32-4d37-a68f-61591097a5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.581312 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d47b0-1f32-4d37-a68f-61591097a5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.582088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1133/2282] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:22.583035 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d47b0-1f32-4d37-a68f-61591097a5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.583275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb31a4a-c5ad-459e-b29a-e80faecafbc5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.592347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d47b0-1f32-4d37-a68f-61591097a5f3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.616690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb31a4a-c5ad-459e-b29a-e80faecafbc5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.620092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1150/2283] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:22.622924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d47b0-1f32-4d37-a68f-61591097a5f3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.628633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4725157-7ac5-43bd-b279-1af94d49c47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.629524 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4725157-7ac5-43bd-b279-1af94d49c47a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.630062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4725157-7ac5-43bd-b279-1af94d49c47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.630508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4725157-7ac5-43bd-b279-1af94d49c47a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.631582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4725157-7ac5-43bd-b279-1af94d49c47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.634224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4725157-7ac5-43bd-b279-1af94d49c47a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.638949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d47b0-1f32-4d37-a68f-61591097a5f3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.650442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6c296-0591-45f3-a114-6370574fcd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.652418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1134/2284] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:22.654146 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4725157-7ac5-43bd-b279-1af94d49c47a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.664970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6c296-0591-45f3-a114-6370574fcd70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.665393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6c296-0591-45f3-a114-6370574fcd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.665767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6c296-0591-45f3-a114-6370574fcd70 admin admin] PATH_INFO: `/v3/projects/1812511b96a64c2f95b2e66892644cec/users/4d2b8b636ffe44a0864f030c10bbfca9/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.683397 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4725157-7ac5-43bd-b279-1af94d49c47a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.700637 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6c296-0591-45f3-a114-6370574fcd70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1812511b96a64c2f95b2e66892644cec, user_id=4d2b8b636ffe44a0864f030c10bbfca9, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.703632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef22cf-5bc1-4a3b-a5b1-9fd158cd5888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.704501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef22cf-5bc1-4a3b-a5b1-9fd158cd5888 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.706329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6c296-0591-45f3-a114-6370574fcd70 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.706856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef22cf-5bc1-4a3b-a5b1-9fd158cd5888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.707196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef22cf-5bc1-4a3b-a5b1-9fd158cd5888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.711934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef22cf-5bc1-4a3b-a5b1-9fd158cd5888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.711934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef22cf-5bc1-4a3b-a5b1-9fd158cd5888 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.736872 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1135/2285] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:22.737889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef22cf-5bc1-4a3b-a5b1-9fd158cd5888 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.738566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1151/2286] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:22 2022] PUT /identity/v3/projects/1812511b96a64c2f95b2e66892644cec/users/4d2b8b636ffe44a0864f030c10bbfca9/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:22.739539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6c296-0591-45f3-a114-6370574fcd70 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.751930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef22cf-5bc1-4a3b-a5b1-9fd158cd5888 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.851055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6c296-0591-45f3-a114-6370574fcd70 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.859498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b36a4b-2b48-46c3-bfb3-9e3fd88600da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.860239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b36a4b-2b48-46c3-bfb3-9e3fd88600da admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.860942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b36a4b-2b48-46c3-bfb3-9e3fd88600da admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.860942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b36a4b-2b48-46c3-bfb3-9e3fd88600da admin admin] PATH_INFO: `/v3/projects/93d64e0a3ca74d278f80c7b680055baa/users/86639ca833cd4fac83c404768dc7ae92/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.885766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b36a4b-2b48-46c3-bfb3-9e3fd88600da admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d64e0a3ca74d278f80c7b680055baa, user_id=86639ca833cd4fac83c404768dc7ae92, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.891589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b36a4b-2b48-46c3-bfb3-9e3fd88600da admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.914862 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1152/2287] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:22 2022] PUT /identity/v3/projects/93d64e0a3ca74d278f80c7b680055baa/users/86639ca833cd4fac83c404768dc7ae92/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:22.915712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b36a4b-2b48-46c3-bfb3-9e3fd88600da admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:22.923922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85d4ce-83c2-4057-a861-c3c5c08c105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.924698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85d4ce-83c2-4057-a861-c3c5c08c105d admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.924982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85d4ce-83c2-4057-a861-c3c5c08c105d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.925258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85d4ce-83c2-4057-a861-c3c5c08c105d admin admin] PATH_INFO: `/v3/projects/94b4f1938e694bd2b7be7c2d40a33d42/users/d3f2a918844e42569e9c89df0f306ccd/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.932238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:22.940900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85d4ce-83c2-4057-a861-c3c5c08c105d admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b4f1938e694bd2b7be7c2d40a33d42, user_id=d3f2a918844e42569e9c89df0f306ccd, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.945092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85d4ce-83c2-4057-a861-c3c5c08c105d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.969236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1136/2288] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:22 2022] PUT /identity/v3/projects/94b4f1938e694bd2b7be7c2d40a33d42/users/d3f2a918844e42569e9c89df0f306ccd/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:22.969236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed247fc-f0d3-4b6c-b957-89389a70c04d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.969236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed247fc-f0d3-4b6c-b957-89389a70c04d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.969236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed247fc-f0d3-4b6c-b957-89389a70c04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.988718 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0ed247fc-f0d3-4b6c-b957-89389a70c04d None None] MFA Rules not processed for user `1de97cce55e944dbb62a920ea11734f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:22.989991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0ed247fc-f0d3-4b6c-b957-89389a70c04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.038316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:23.077721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b36a4b-2b48-46c3-bfb3-9e3fd88600da admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.156556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1137/2289] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:23.158230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f445ffd-4359-483e-b726-d11020cf1c79 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.158818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f445ffd-4359-483e-b726-d11020cf1c79 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.159229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f445ffd-4359-483e-b726-d11020cf1c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.175407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2af39-104d-4558-90cd-64be60672788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.176147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2af39-104d-4558-90cd-64be60672788 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.176440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2af39-104d-4558-90cd-64be60672788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.178196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2af39-104d-4558-90cd-64be60672788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.180947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2af39-104d-4558-90cd-64be60672788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:23.184978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2af39-104d-4558-90cd-64be60672788 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:23.209856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8f445ffd-4359-483e-b726-d11020cf1c79 None None] MFA Rules not processed for user `86639ca833cd4fac83c404768dc7ae92`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:23.212251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8f445ffd-4359-483e-b726-d11020cf1c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.216281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1153/2290] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:23.217384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47c331-4bae-4a02-b847-9e1dd3098761 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.217678 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47c331-4bae-4a02-b847-9e1dd3098761 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.218013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47c331-4bae-4a02-b847-9e1dd3098761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.273740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fc47c331-4bae-4a02-b847-9e1dd3098761 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:23.276971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fc47c331-4bae-4a02-b847-9e1dd3098761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.300980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1154/2291] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:23.304373 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47c331-4bae-4a02-b847-9e1dd3098761 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:23.310221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fc47c331-4bae-4a02-b847-9e1dd3098761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.338495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47c331-4bae-4a02-b847-9e1dd3098761 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.353116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.353942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.354396 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.355259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] PATH_INFO: `/v3/projects/e75f6db00cf340788a011d7daee52d74` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.356731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e75f6db00cf340788a011d7daee52d74)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:23.366522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:23.385301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1138/2292] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:23.386426 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f445ffd-4359-483e-b726-d11020cf1c79 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:23.425173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f445ffd-4359-483e-b726-d11020cf1c79 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.439122 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ed81e-e755-4087-9c22-4fe3a59766b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.440293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ed81e-e755-4087-9c22-4fe3a59766b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.440671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ed81e-e755-4087-9c22-4fe3a59766b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.441066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ed81e-e755-4087-9c22-4fe3a59766b7 admin admin] PATH_INFO: `/v3/projects/1812511b96a64c2f95b2e66892644cec/users/4d2b8b636ffe44a0864f030c10bbfca9/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.445131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ed81e-e755-4087-9c22-4fe3a59766b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1812511b96a64c2f95b2e66892644cec, user_id=4d2b8b636ffe44a0864f030c10bbfca9, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:23.448016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ed81e-e755-4087-9c22-4fe3a59766b7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:23.478635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e75f6db00cf340788a011d7daee52d74', 'request_id': 'req-dcc75821-7a53-4390-825f-4b0fd77409e6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.479732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1139/2293] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:23 2022] PUT /identity/v3/projects/1812511b96a64c2f95b2e66892644cec/users/4d2b8b636ffe44a0864f030c10bbfca9/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:23.480967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ed81e-e755-4087-9c22-4fe3a59766b7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:23.500755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e75f6db00cf340788a011d7daee52d74', 'request_id': 'req-dcc75821-7a53-4390-825f-4b0fd77409e6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.518591 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e75f6db00cf340788a011d7daee52d74', 'request_id': 'req-dcc75821-7a53-4390-825f-4b0fd77409e6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.540520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] The token cache is being invalidate because project e75f6db00cf340788a011d7daee52d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:23.540824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.543514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.545509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1155/2294] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:23 2022] DELETE /identity/v3/projects/e75f6db00cf340788a011d7daee52d74 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:23.546277 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:23.549901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.631138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ed81e-e755-4087-9c22-4fe3a59766b7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.642486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1655c3-5280-4900-b8ac-b031cb8ee1e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.643472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1655c3-5280-4900-b8ac-b031cb8ee1e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.643595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1655c3-5280-4900-b8ac-b031cb8ee1e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.643732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:23.644247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1655c3-5280-4900-b8ac-b031cb8ee1e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.651169 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4f1655c3-5280-4900-b8ac-b031cb8ee1e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.741756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75821-7a53-4390-825f-4b0fd77409e6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.824312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:23.854194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1655c3-5280-4900-b8ac-b031cb8ee1e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:23.856273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1655c3-5280-4900-b8ac-b031cb8ee1e4 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:23.876181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99964d-827b-40e1-b1e3-dad7a629e5b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '794dbf38682e4b4f939854eecbacaf07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.876664 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1140/2295] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:23.877238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1655c3-5280-4900-b8ac-b031cb8ee1e4 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:23.877607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99964d-827b-40e1-b1e3-dad7a629e5b8 service glance] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.883665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4f1655c3-5280-4900-b8ac-b031cb8ee1e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.900470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99964d-827b-40e1-b1e3-dad7a629e5b8 service glance] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.902360 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99964d-827b-40e1-b1e3-dad7a629e5b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.910498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8e99964d-827b-40e1-b1e3-dad7a629e5b8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.915628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1655c3-5280-4900-b8ac-b031cb8ee1e4 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.931089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0ae29-fe75-4096-af46-2f919e44667a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.931857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0ae29-fe75-4096-af46-2f919e44667a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.932224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0ae29-fe75-4096-af46-2f919e44667a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.932546 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0ae29-fe75-4096-af46-2f919e44667a admin admin] PATH_INFO: `/v3/users/91a72ae9525244abaf8f760938c4f3b5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.935494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0ae29-fe75-4096-af46-2f919e44667a admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a72ae9525244abaf8f760938c4f3b5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:23.937578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0ae29-fe75-4096-af46-2f919e44667a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:24.063942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3be0ae29-fe75-4096-af46-2f919e44667a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a72ae9525244abaf8f760938c4f3b5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.066902 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3be0ae29-fe75-4096-af46-2f919e44667a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a72ae9525244abaf8f760938c4f3b5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.067728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3be0ae29-fe75-4096-af46-2f919e44667a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a72ae9525244abaf8f760938c4f3b5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.085785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3be0ae29-fe75-4096-af46-2f919e44667a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a72ae9525244abaf8f760938c4f3b5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.105046 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3be0ae29-fe75-4096-af46-2f919e44667a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a72ae9525244abaf8f760938c4f3b5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.106382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99964d-827b-40e1-b1e3-dad7a629e5b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:24.112323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99964d-827b-40e1-b1e3-dad7a629e5b8 service glance] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:24.141814 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1156/2296] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:24.148284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1141/2297] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:23 2022] DELETE /identity/v3/users/91a72ae9525244abaf8f760938c4f3b5 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:25.271406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99964d-827b-40e1-b1e3-dad7a629e5b8 service glance] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:25.286792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99964d-827b-40e1-b1e3-dad7a629e5b8 service glance] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:25.500569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678340f5-281b-4f31-a309-434091eda47b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:25.501085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678340f5-281b-4f31-a309-434091eda47b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:25.501203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678340f5-281b-4f31-a309-434091eda47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:25.564642 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:25.565743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:25.566229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:25.567074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] PATH_INFO: `/v3/projects/742981a8e4df47c7b78e0ebf00fdec5e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:25.571334 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] RBAC: Authorizing `identity:delete_project(project_id=742981a8e4df47c7b78e0ebf00fdec5e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:25.574042 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:25.588591 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-678340f5-281b-4f31-a309-434091eda47b None None] MFA Rules not processed for user `d3f2a918844e42569e9c89df0f306ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:25.591790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-678340f5-281b-4f31-a309-434091eda47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:25.657497 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:25.686798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '742981a8e4df47c7b78e0ebf00fdec5e', 'request_id': 'req-5d64aca9-297f-4f3a-b328-56577511d10c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.713973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '742981a8e4df47c7b78e0ebf00fdec5e', 'request_id': 'req-5d64aca9-297f-4f3a-b328-56577511d10c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.739971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1142/2298] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:25.740735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '742981a8e4df47c7b78e0ebf00fdec5e', 'request_id': 'req-5d64aca9-297f-4f3a-b328-56577511d10c'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.741242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678340f5-281b-4f31-a309-434091eda47b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:25.744356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-678340f5-281b-4f31-a309-434091eda47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:25.762942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678340f5-281b-4f31-a309-434091eda47b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:25.769351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] The token cache is being invalidate because project 742981a8e4df47c7b78e0ebf00fdec5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:25.769351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.769351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5d64aca9-297f-4f3a-b328-56577511d10c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.775988 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1157/2299] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:25 2022] DELETE /identity/v3/projects/742981a8e4df47c7b78e0ebf00fdec5e => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:25.778057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939a8c7-8219-40cc-81a0-0c9e1be4d65f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:25.778359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939a8c7-8219-40cc-81a0-0c9e1be4d65f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:25.778633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939a8c7-8219-40cc-81a0-0c9e1be4d65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:25.816996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6939a8c7-8219-40cc-81a0-0c9e1be4d65f None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:25.818220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6939a8c7-8219-40cc-81a0-0c9e1be4d65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:25.876663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:25.907208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085d51b-d2d7-44c4-9cd4-4a3a8f9724bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:25.907871 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085d51b-d2d7-44c4-9cd4-4a3a8f9724bf admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:25.908562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085d51b-d2d7-44c4-9cd4-4a3a8f9724bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:25.908562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085d51b-d2d7-44c4-9cd4-4a3a8f9724bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:25.909211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085d51b-d2d7-44c4-9cd4-4a3a8f9724bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:25.913930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085d51b-d2d7-44c4-9cd4-4a3a8f9724bf admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:25.978501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1143/2300] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:25 2022] POST /identity/v3/projects => generated 371 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:25.985019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085d51b-d2d7-44c4-9cd4-4a3a8f9724bf admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:25.987652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6085d51b-d2d7-44c4-9cd4-4a3a8f9724bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:26.003701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1158/2301] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:26.014041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73249580-6272-4925-9c44-a2464724cddb None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.014352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73249580-6272-4925-9c44-a2464724cddb None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.014908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73249580-6272-4925-9c44-a2464724cddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.088453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-73249580-6272-4925-9c44-a2464724cddb None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:26.093035 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-73249580-6272-4925-9c44-a2464724cddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:26.241745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085d51b-d2d7-44c4-9cd4-4a3a8f9724bf admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.254837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c2aed-e65e-4137-a49d-e6dc619c6d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.255532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c2aed-e65e-4137-a49d-e6dc619c6d02 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.255821 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c2aed-e65e-4137-a49d-e6dc619c6d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.256133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c2aed-e65e-4137-a49d-e6dc619c6d02 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.256794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c2aed-e65e-4137-a49d-e6dc619c6d02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.263498 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c2aed-e65e-4137-a49d-e6dc619c6d02 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.290443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1159/2302] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:26.300080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73249580-6272-4925-9c44-a2464724cddb None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.323996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73249580-6272-4925-9c44-a2464724cddb None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.333263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c6d31-969f-4215-bcfa-e04aca6633b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.333809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c6d31-969f-4215-bcfa-e04aca6633b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.334090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c6d31-969f-4215-bcfa-e04aca6633b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.334569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c6d31-969f-4215-bcfa-e04aca6633b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.335410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c6d31-969f-4215-bcfa-e04aca6633b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:26.335888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c6d31-969f-4215-bcfa-e04aca6633b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.338532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c6d31-969f-4215-bcfa-e04aca6633b0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.341314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1144/2303] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:25 2022] POST /identity/v3/users => generated 292 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:26.349939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c2aed-e65e-4137-a49d-e6dc619c6d02 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.360292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1160/2304] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:26.370636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c6d31-969f-4215-bcfa-e04aca6633b0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.378282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c2aed-e65e-4137-a49d-e6dc619c6d02 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.384741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c6d31-969f-4215-bcfa-e04aca6633b0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.391470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ae225-8a4c-4fc2-a0f8-6cd1d29ed7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.392763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ae225-8a4c-4fc2-a0f8-6cd1d29ed7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.393034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ae225-8a4c-4fc2-a0f8-6cd1d29ed7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.393309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ae225-8a4c-4fc2-a0f8-6cd1d29ed7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.393981 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ae225-8a4c-4fc2-a0f8-6cd1d29ed7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.395471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36238c52-35b6-418a-96e1-600d5aa694aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.396265 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36238c52-35b6-418a-96e1-600d5aa694aa admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.396695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36238c52-35b6-418a-96e1-600d5aa694aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.396974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36238c52-35b6-418a-96e1-600d5aa694aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.397615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36238c52-35b6-418a-96e1-600d5aa694aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.399133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ae225-8a4c-4fc2-a0f8-6cd1d29ed7a5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.401922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36238c52-35b6-418a-96e1-600d5aa694aa admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.425529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1145/2305] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:26.426449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ae225-8a4c-4fc2-a0f8-6cd1d29ed7a5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.447735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ae225-8a4c-4fc2-a0f8-6cd1d29ed7a5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.471605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1161/2306] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:26 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:26.471931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127f30b-acd1-411b-8cd6-c69808f53cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.471931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127f30b-acd1-411b-8cd6-c69808f53cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.471931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127f30b-acd1-411b-8cd6-c69808f53cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.471931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127f30b-acd1-411b-8cd6-c69808f53cb2 admin admin] PATH_INFO: `/v3/projects/e442497bec93476597f27e3478edf460/users/e8bfde648b604a2480767fe9b337d546/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.472650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36238c52-35b6-418a-96e1-600d5aa694aa admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.511937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127f30b-acd1-411b-8cd6-c69808f53cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e442497bec93476597f27e3478edf460, user_id=e8bfde648b604a2480767fe9b337d546, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.514699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127f30b-acd1-411b-8cd6-c69808f53cb2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.536247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:26.574521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1146/2307] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:26 2022] PUT /identity/v3/projects/e442497bec93476597f27e3478edf460/users/e8bfde648b604a2480767fe9b337d546/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:26.575996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127f30b-acd1-411b-8cd6-c69808f53cb2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.658616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36238c52-35b6-418a-96e1-600d5aa694aa admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.675984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127f30b-acd1-411b-8cd6-c69808f53cb2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.681625 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcaa5a-707f-4e8b-b2f5-861a4ae69476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.682407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcaa5a-707f-4e8b-b2f5-861a4ae69476 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.682822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcaa5a-707f-4e8b-b2f5-861a4ae69476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.683102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcaa5a-707f-4e8b-b2f5-861a4ae69476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.683886 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dcaa5a-707f-4e8b-b2f5-861a4ae69476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.686060 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dcaa5a-707f-4e8b-b2f5-861a4ae69476 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.726464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1147/2308] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:26.727741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcaa5a-707f-4e8b-b2f5-861a4ae69476 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.739953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcaa5a-707f-4e8b-b2f5-861a4ae69476 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.747268 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbef958-0901-445a-bb76-723c493d6f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.748110 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbef958-0901-445a-bb76-723c493d6f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.748637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbef958-0901-445a-bb76-723c493d6f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.749006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbef958-0901-445a-bb76-723c493d6f8f admin admin] PATH_INFO: `/v3/projects/e442497bec93476597f27e3478edf460/users/e8bfde648b604a2480767fe9b337d546/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.752034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbef958-0901-445a-bb76-723c493d6f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e442497bec93476597f27e3478edf460, user_id=e8bfde648b604a2480767fe9b337d546, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.756467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbef958-0901-445a-bb76-723c493d6f8f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.792018 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560346-5681-40b2-987a-e511d721d87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.792857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560346-5681-40b2-987a-e511d721d87f admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.793243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560346-5681-40b2-987a-e511d721d87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.793589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560346-5681-40b2-987a-e511d721d87f admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.794523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca560346-5681-40b2-987a-e511d721d87f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.797917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca560346-5681-40b2-987a-e511d721d87f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.815819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1148/2309] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:26 2022] PUT /identity/v3/projects/e442497bec93476597f27e3478edf460/users/e8bfde648b604a2480767fe9b337d546/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:26.875139 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1162/2310] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:26 2022] POST /identity/v3/users => generated 287 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:26.882700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbef958-0901-445a-bb76-723c493d6f8f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.939002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:27.099640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbef958-0901-445a-bb76-723c493d6f8f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:27.118250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f137302-c145-4934-bd0c-f5980585e27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:27.119012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137302-c145-4934-bd0c-f5980585e27f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.119350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137302-c145-4934-bd0c-f5980585e27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.120116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137302-c145-4934-bd0c-f5980585e27f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.120397 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f137302-c145-4934-bd0c-f5980585e27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.126326 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f137302-c145-4934-bd0c-f5980585e27f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.156803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1149/2311] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:27.157824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560346-5681-40b2-987a-e511d721d87f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:27.174341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560346-5681-40b2-987a-e511d721d87f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:27.192282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab8662-e1e9-4198-a9ab-013c51d2b285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:27.196878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab8662-e1e9-4198-a9ab-013c51d2b285 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.198138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab8662-e1e9-4198-a9ab-013c51d2b285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.198630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab8662-e1e9-4198-a9ab-013c51d2b285 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.206687 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab8662-e1e9-4198-a9ab-013c51d2b285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.214685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab8662-e1e9-4198-a9ab-013c51d2b285 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.244377 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f137302-c145-4934-bd0c-f5980585e27f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:27.246941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1163/2312] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:27.269644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f137302-c145-4934-bd0c-f5980585e27f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:27.284470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804c3e6-b4e0-4ed0-9b1e-4015a149f10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:27.287027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804c3e6-b4e0-4ed0-9b1e-4015a149f10b admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.287484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804c3e6-b4e0-4ed0-9b1e-4015a149f10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.287913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804c3e6-b4e0-4ed0-9b1e-4015a149f10b admin admin] PATH_INFO: `/v3/projects/f12553f79dbd4d75a291d3a07a8dbbba/users/731cf6df5bda42148e1420e365306445/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.322924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804c3e6-b4e0-4ed0-9b1e-4015a149f10b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12553f79dbd4d75a291d3a07a8dbbba, user_id=731cf6df5bda42148e1420e365306445, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.341226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804c3e6-b4e0-4ed0-9b1e-4015a149f10b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.374934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1150/2313] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:27 2022] PUT /identity/v3/projects/f12553f79dbd4d75a291d3a07a8dbbba/users/731cf6df5bda42148e1420e365306445/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:27.379912 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab318da5-c03c-4dda-ac8e-6ad891317d65 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.380440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab318da5-c03c-4dda-ac8e-6ad891317d65 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.380533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab318da5-c03c-4dda-ac8e-6ad891317d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.459803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ab318da5-c03c-4dda-ac8e-6ad891317d65 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:27.461250 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ab318da5-c03c-4dda-ac8e-6ad891317d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:27.668160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1164/2314] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:27.721411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804c3e6-b4e0-4ed0-9b1e-4015a149f10b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:27.723259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2804c3e6-b4e0-4ed0-9b1e-4015a149f10b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:27.864369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804c3e6-b4e0-4ed0-9b1e-4015a149f10b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:27.869231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39936a4-f06a-438f-971d-6ae2ba6681fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:27.870256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39936a4-f06a-438f-971d-6ae2ba6681fc service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.870544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39936a4-f06a-438f-971d-6ae2ba6681fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.870872 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39936a4-f06a-438f-971d-6ae2ba6681fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.885194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39936a4-f06a-438f-971d-6ae2ba6681fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.900689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39936a4-f06a-438f-971d-6ae2ba6681fc service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.918374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1151/2315] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:28.347065 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be584d07-b626-43f2-ba4a-1d63e8460ffa None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:28.347366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be584d07-b626-43f2-ba4a-1d63e8460ffa None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:28.348155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be584d07-b626-43f2-ba4a-1d63e8460ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:28.426408 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-be584d07-b626-43f2-ba4a-1d63e8460ffa None None] MFA Rules not processed for user `e8bfde648b604a2480767fe9b337d546`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:28.427955 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-be584d07-b626-43f2-ba4a-1d63e8460ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:28.519218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:28.599405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1165/2316] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:28.611547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39936a4-f06a-438f-971d-6ae2ba6681fc service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:28.612967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b39936a4-f06a-438f-971d-6ae2ba6681fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:28.702161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39936a4-f06a-438f-971d-6ae2ba6681fc service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:28.706732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4be32-7d3c-40e8-9b29-2a3d19554e4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:28.707727 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4be32-7d3c-40e8-9b29-2a3d19554e4a service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:28.708400 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4be32-7d3c-40e8-9b29-2a3d19554e4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:28.709028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4be32-7d3c-40e8-9b29-2a3d19554e4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:28.711969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9fc4be32-7d3c-40e8-9b29-2a3d19554e4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:28.881381 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:28.959200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4be32-7d3c-40e8-9b29-2a3d19554e4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:28.960703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4be32-7d3c-40e8-9b29-2a3d19554e4a service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:28.973949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1152/2317] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:30.079050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6019ffb-3877-44a2-8e76-7c23c67a17e5 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:30.080573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6019ffb-3877-44a2-8e76-7c23c67a17e5 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:30.081133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6019ffb-3877-44a2-8e76-7c23c67a17e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:30.142980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f6019ffb-3877-44a2-8e76-7c23c67a17e5 None None] MFA Rules not processed for user `731cf6df5bda42148e1420e365306445`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:30.144440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f6019ffb-3877-44a2-8e76-7c23c67a17e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:30.256780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1166/2318] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:30.265188 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4be32-7d3c-40e8-9b29-2a3d19554e4a service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:30.266759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9fc4be32-7d3c-40e8-9b29-2a3d19554e4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:30.281282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4be32-7d3c-40e8-9b29-2a3d19554e4a service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:30.287921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc49d0c-99da-4664-acbe-0cb12b577084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:30.288863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49d0c-99da-4664-acbe-0cb12b577084 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:30.289302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49d0c-99da-4664-acbe-0cb12b577084 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:30.289760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49d0c-99da-4664-acbe-0cb12b577084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:30.301432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc49d0c-99da-4664-acbe-0cb12b577084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:30.303109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc49d0c-99da-4664-acbe-0cb12b577084 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:30.316386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1153/2319] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:30.831693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6019ffb-3877-44a2-8e76-7c23c67a17e5 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:30.846641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6019ffb-3877-44a2-8e76-7c23c67a17e5 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:30.853129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22782ac9-9482-4a55-8bc1-bbe43e997fb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:30.853978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22782ac9-9482-4a55-8bc1-bbe43e997fb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:30.854447 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22782ac9-9482-4a55-8bc1-bbe43e997fb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:30.854855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22782ac9-9482-4a55-8bc1-bbe43e997fb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:30.880507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22782ac9-9482-4a55-8bc1-bbe43e997fb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:30.884245 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22782ac9-9482-4a55-8bc1-bbe43e997fb9 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:30.921410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1167/2320] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:32.150724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d7c84-ae77-4de5-8ad6-d509a1568127 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:32.153690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d7c84-ae77-4de5-8ad6-d509a1568127 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:32.155444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d7c84-ae77-4de5-8ad6-d509a1568127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:32.189929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9b8d7c84-ae77-4de5-8ad6-d509a1568127 None None] MFA Rules not processed for user `731cf6df5bda42148e1420e365306445`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:32.191472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9b8d7c84-ae77-4de5-8ad6-d509a1568127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:32.206277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1154/2321] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:34.531530 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029359f-2e4f-438e-8833-703fa4c325c1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:34.532133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029359f-2e4f-438e-8833-703fa4c325c1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:34.532441 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029359f-2e4f-438e-8833-703fa4c325c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:34.632678 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d029359f-2e4f-438e-8833-703fa4c325c1 None None] MFA Rules not processed for user `4d2b8b636ffe44a0864f030c10bbfca9`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:34.635050 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d029359f-2e4f-438e-8833-703fa4c325c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:34.826227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1168/2322] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:34.868828 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d7c84-ae77-4de5-8ad6-d509a1568127 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:34.871798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9b8d7c84-ae77-4de5-8ad6-d509a1568127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:34.968870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d7c84-ae77-4de5-8ad6-d509a1568127 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:34.975956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bf1cc-a3a5-40b2-895c-23f014919e6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:34.976612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bf1cc-a3a5-40b2-895c-23f014919e6b service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:34.976898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bf1cc-a3a5-40b2-895c-23f014919e6b service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:34.977174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bf1cc-a3a5-40b2-895c-23f014919e6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:34.978957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e81bf1cc-a3a5-40b2-895c-23f014919e6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:35.099344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bf1cc-a3a5-40b2-895c-23f014919e6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:35.101954 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bf1cc-a3a5-40b2-895c-23f014919e6b service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:35.115464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1155/2323] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:37.095005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40248-dacc-45e6-80b8-9fb31c36a458 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:37.095505 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40248-dacc-45e6-80b8-9fb31c36a458 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:37.095505 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40248-dacc-45e6-80b8-9fb31c36a458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:37.133370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-c7b40248-dacc-45e6-80b8-9fb31c36a458 None None] MFA Rules not processed for user `1c665588c50d47dbaf1f8fde082ee73a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:37.134761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c7b40248-dacc-45e6-80b8-9fb31c36a458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:37.156610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1169/2324] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:40.772850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bf1cc-a3a5-40b2-895c-23f014919e6b service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:40.778654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e81bf1cc-a3a5-40b2-895c-23f014919e6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:40.802461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bf1cc-a3a5-40b2-895c-23f014919e6b service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:40.812164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:40.813008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:40.813438 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:40.813861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] PATH_INFO: `/v3/users/a99c0bc8fd2443dbaa41d47b7e5c22ab` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:40.815478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a99c0bc8fd2443dbaa41d47b7e5c22ab)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:40.819174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:40.966442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a99c0bc8fd2443dbaa41d47b7e5c22ab'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:40.967138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a99c0bc8fd2443dbaa41d47b7e5c22ab'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:40.967754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a99c0bc8fd2443dbaa41d47b7e5c22ab'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:40.979960 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a99c0bc8fd2443dbaa41d47b7e5c22ab'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:40.987008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:40.997482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a99c0bc8fd2443dbaa41d47b7e5c22ab'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:41.025309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1156/2325] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:40 2022] DELETE /identity/v3/users/a99c0bc8fd2443dbaa41d47b7e5c22ab => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:41.030437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae5016-e68a-4e9e-85dd-f7c995cd02ad None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:41.030957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae5016-e68a-4e9e-85dd-f7c995cd02ad None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:41.031177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae5016-e68a-4e9e-85dd-f7c995cd02ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:41.051570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-80ae5016-e68a-4e9e-85dd-f7c995cd02ad None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:41.052920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-80ae5016-e68a-4e9e-85dd-f7c995cd02ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:41.077212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:41.140822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1170/2326] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:41.153442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:41.153874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:41.217792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49486fa2-445e-41ea-a572-9dc34064f72b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:41.221838 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e87b0a-292a-449f-918a-a165a9fbf5bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:41.222470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87b0a-292a-449f-918a-a165a9fbf5bc service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:41.222747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87b0a-292a-449f-918a-a165a9fbf5bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:41.223018 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87b0a-292a-449f-918a-a165a9fbf5bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:41.234250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e87b0a-292a-449f-918a-a165a9fbf5bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:41.235780 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e87b0a-292a-449f-918a-a165a9fbf5bc service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:41.254577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1157/2327] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:41.642543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae5016-e68a-4e9e-85dd-f7c995cd02ad None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:41.644630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-80ae5016-e68a-4e9e-85dd-f7c995cd02ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:41.658476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae5016-e68a-4e9e-85dd-f7c995cd02ad None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:41.664194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:41.664842 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:41.665130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:41.665407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] PATH_INFO: `/v3/projects/7c003a4052364861a10ac993a3e58ad2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:41.666680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c003a4052364861a10ac993a3e58ad2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:41.669167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:41.728974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c003a4052364861a10ac993a3e58ad2', 'request_id': 'req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:41.738320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c003a4052364861a10ac993a3e58ad2', 'request_id': 'req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:41.745786 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c003a4052364861a10ac993a3e58ad2', 'request_id': 'req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:41.762224 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] The token cache is being invalidate because project 7c003a4052364861a10ac993a3e58ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:41.763592 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:41.764964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-10e5b1d7-ab27-4da4-81d4-ba52c41c5a6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:41.767054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1171/2328] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:41 2022] DELETE /identity/v3/projects/7c003a4052364861a10ac993a3e58ad2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:41.771788 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273935d-3991-448d-8c48-b5fdfa51bf89 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:41.772091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273935d-3991-448d-8c48-b5fdfa51bf89 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:41.772359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273935d-3991-448d-8c48-b5fdfa51bf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:41.805417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7273935d-3991-448d-8c48-b5fdfa51bf89 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:41.807304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7273935d-3991-448d-8c48-b5fdfa51bf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:41.890436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1158/2329] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:41.892698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27008749-e9a0-4b0f-894c-011bb2efbd3f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:41.892997 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27008749-e9a0-4b0f-894c-011bb2efbd3f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:41.893338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27008749-e9a0-4b0f-894c-011bb2efbd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:41.923948 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-27008749-e9a0-4b0f-894c-011bb2efbd3f None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:41.925948 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-27008749-e9a0-4b0f-894c-011bb2efbd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:41.950241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1172/2330] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:41.951708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273935d-3991-448d-8c48-b5fdfa51bf89 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:41.978368 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273935d-3991-448d-8c48-b5fdfa51bf89 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:41.988833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bd01-d2be-47de-8490-5987c6a97255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:41.989451 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bd01-d2be-47de-8490-5987c6a97255 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:41.989731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bd01-d2be-47de-8490-5987c6a97255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:41.990005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bd01-d2be-47de-8490-5987c6a97255 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:41.990746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bd01-d2be-47de-8490-5987c6a97255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:41.990961 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bd01-d2be-47de-8490-5987c6a97255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:41.992842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bd01-d2be-47de-8490-5987c6a97255 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:42.012835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1159/2331] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:42.014082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27008749-e9a0-4b0f-894c-011bb2efbd3f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:42.034120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27008749-e9a0-4b0f-894c-011bb2efbd3f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:42.043556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9281d5-9814-40f7-8902-a75e6b409783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:42.044219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9281d5-9814-40f7-8902-a75e6b409783 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:42.044502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9281d5-9814-40f7-8902-a75e6b409783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:42.044779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9281d5-9814-40f7-8902-a75e6b409783 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:42.045417 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9281d5-9814-40f7-8902-a75e6b409783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:42.047504 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9281d5-9814-40f7-8902-a75e6b409783 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:42.083370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bd01-d2be-47de-8490-5987c6a97255 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:42.085516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1173/2332] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:42 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:42.097880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bd01-d2be-47de-8490-5987c6a97255 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:42.202466 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1294ea-f6e5-449c-a8a3-3a0d3a40c59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:42.205347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1294ea-f6e5-449c-a8a3-3a0d3a40c59e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:42.205347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1294ea-f6e5-449c-a8a3-3a0d3a40c59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:42.205650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1294ea-f6e5-449c-a8a3-3a0d3a40c59e admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:42.206448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1294ea-f6e5-449c-a8a3-3a0d3a40c59e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:42.213576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1294ea-f6e5-449c-a8a3-3a0d3a40c59e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:42.227160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:42.249451 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1160/2333] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:42 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:42.252790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9281d5-9814-40f7-8902-a75e6b409783 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:42.284158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9281d5-9814-40f7-8902-a75e6b409783 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:42.295323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0776f21-18db-4e35-86b2-b5658b67dd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:42.297876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0776f21-18db-4e35-86b2-b5658b67dd54 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:42.300886 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0776f21-18db-4e35-86b2-b5658b67dd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:42.301781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0776f21-18db-4e35-86b2-b5658b67dd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:42.303155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0776f21-18db-4e35-86b2-b5658b67dd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:42.307317 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0776f21-18db-4e35-86b2-b5658b67dd54 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:42.343601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1174/2334] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:42.344164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1294ea-f6e5-449c-a8a3-3a0d3a40c59e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:42.365270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1294ea-f6e5-449c-a8a3-3a0d3a40c59e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:42.377109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7df5b-103b-4e9b-8061-802f236cc5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:42.378017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7df5b-103b-4e9b-8061-802f236cc5cd admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:42.378556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7df5b-103b-4e9b-8061-802f236cc5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:42.379267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7df5b-103b-4e9b-8061-802f236cc5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:42.380379 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7df5b-103b-4e9b-8061-802f236cc5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:42.382408 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7df5b-103b-4e9b-8061-802f236cc5cd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:42.397609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1161/2335] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:42.402873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0776f21-18db-4e35-86b2-b5658b67dd54 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:42.418938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0776f21-18db-4e35-86b2-b5658b67dd54 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:42.425279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e45fa-fe21-4400-900f-254187723f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:42.426542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e45fa-fe21-4400-900f-254187723f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:42.426996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e45fa-fe21-4400-900f-254187723f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:42.427426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e45fa-fe21-4400-900f-254187723f61 admin admin] PATH_INFO: `/v3/projects/b862d2489d67446ab37c5f9d4b3322d2/users/952e091d7ce94adab3af672e010b6259/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:42.465955 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e45fa-fe21-4400-900f-254187723f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b862d2489d67446ab37c5f9d4b3322d2, user_id=952e091d7ce94adab3af672e010b6259, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:42.469246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e45fa-fe21-4400-900f-254187723f61 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:42.487640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1175/2336] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:42 2022] PUT /identity/v3/projects/b862d2489d67446ab37c5f9d4b3322d2/users/952e091d7ce94adab3af672e010b6259/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:42.500079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948d17b-87db-4b75-8ae5-6c65afecd966 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:42.500417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948d17b-87db-4b75-8ae5-6c65afecd966 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:42.500656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948d17b-87db-4b75-8ae5-6c65afecd966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:42.556271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6948d17b-87db-4b75-8ae5-6c65afecd966 None None] MFA Rules not processed for user `952e091d7ce94adab3af672e010b6259`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:42.556271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6948d17b-87db-4b75-8ae5-6c65afecd966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:42.656984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1162/2337] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:42.672822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e45fa-fe21-4400-900f-254187723f61 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:42.674751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b11e45fa-fe21-4400-900f-254187723f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:42.763780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:42.773482 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e45fa-fe21-4400-900f-254187723f61 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:42.779280 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9a03f-3dd4-4c1e-87b0-cccaecba7fe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:42.780066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9a03f-3dd4-4c1e-87b0-cccaecba7fe7 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:42.780788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9a03f-3dd4-4c1e-87b0-cccaecba7fe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:42.781354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9a03f-3dd4-4c1e-87b0-cccaecba7fe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:42.792675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9a03f-3dd4-4c1e-87b0-cccaecba7fe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:42.794553 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9a03f-3dd4-4c1e-87b0-cccaecba7fe7 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:42.806378 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1176/2338] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:46.505953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6b3c7-1f03-4f93-b084-5bda8048407c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:46.506993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6b3c7-1f03-4f93-b084-5bda8048407c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:46.506993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6b3c7-1f03-4f93-b084-5bda8048407c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:46.508906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1163/2339] 10.222.0.86 () {54 vars in 921 bytes} [Wed Aug 24 10:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:04:46.515938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55a751-81ae-4b59-be21-ed84541b620a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:46.516470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55a751-81ae-4b59-be21-ed84541b620a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:46.516552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55a751-81ae-4b59-be21-ed84541b620a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:46.538505 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ae55a751-81ae-4b59-be21-ed84541b620a None None] MFA Rules not processed for user `ef8d30e87c624d6180656d998afcebe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:46.539945 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ae55a751-81ae-4b59-be21-ed84541b620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:46.619022 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1177/2340] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Aug 24 10:04:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:46.630431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6b3c7-1f03-4f93-b084-5bda8048407c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:46.632189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1be6b3c7-1f03-4f93-b084-5bda8048407c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:46.648863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6b3c7-1f03-4f93-b084-5bda8048407c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:46.654501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f3800-9e90-49ce-b2f4-a9847431b11a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:46.655105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f3800-9e90-49ce-b2f4-a9847431b11a service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:46.655388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f3800-9e90-49ce-b2f4-a9847431b11a service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:46.655668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f3800-9e90-49ce-b2f4-a9847431b11a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:46.665261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f3800-9e90-49ce-b2f4-a9847431b11a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:46.667452 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f3800-9e90-49ce-b2f4-a9847431b11a service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:46.679494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1164/2341] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:47.191651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55a751-81ae-4b59-be21-ed84541b620a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:47.193306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ae55a751-81ae-4b59-be21-ed84541b620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:47.280014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55a751-81ae-4b59-be21-ed84541b620a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:47.284621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:47.285246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:47.285525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:47.285799 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] PATH_INFO: `/v3/users/d3f2a918844e42569e9c89df0f306ccd` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:47.286923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3f2a918844e42569e9c89df0f306ccd)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:47.288901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:47.338475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3f2a918844e42569e9c89df0f306ccd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.339131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3f2a918844e42569e9c89df0f306ccd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.339733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3f2a918844e42569e9c89df0f306ccd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.347395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3f2a918844e42569e9c89df0f306ccd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.357075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3f2a918844e42569e9c89df0f306ccd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.378274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1178/2342] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:47 2022] DELETE /identity/v3/users/d3f2a918844e42569e9c89df0f306ccd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:47.624833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f3800-9e90-49ce-b2f4-a9847431b11a service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:47.636842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f3800-9e90-49ce-b2f4-a9847431b11a service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:47.730582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:47.731239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:47.731519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:47.731794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] PATH_INFO: `/v3/projects/94b4f1938e694bd2b7be7c2d40a33d42` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:47.733337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=94b4f1938e694bd2b7be7c2d40a33d42)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:47.735532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:47.797637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94b4f1938e694bd2b7be7c2d40a33d42', 'request_id': 'req-f177709a-5d50-4073-87d0-795b14e81b6c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.806473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94b4f1938e694bd2b7be7c2d40a33d42', 'request_id': 'req-f177709a-5d50-4073-87d0-795b14e81b6c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.811929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=1,keystone:SELECT=93,keystone:DELETE=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:47.813244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94b4f1938e694bd2b7be7c2d40a33d42', 'request_id': 'req-f177709a-5d50-4073-87d0-795b14e81b6c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.827410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] The token cache is being invalidate because project 94b4f1938e694bd2b7be7c2d40a33d42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:47.827700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.828368 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.829862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1165/2343] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:47 2022] DELETE /identity/v3/projects/94b4f1938e694bd2b7be7c2d40a33d42 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:47.833880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:47.836498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:47.848623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:47.852965 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8aa95-b137-473b-a1e6-15f9871a6cb0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:47.920594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48308913-8a36-4e3b-afc2-fd4a62e53a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:47.921513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48308913-8a36-4e3b-afc2-fd4a62e53a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:47.921797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48308913-8a36-4e3b-afc2-fd4a62e53a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:47.922073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48308913-8a36-4e3b-afc2-fd4a62e53a50 admin admin] PATH_INFO: `/v3/users/e8bfde648b604a2480767fe9b337d546` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:47.923205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48308913-8a36-4e3b-afc2-fd4a62e53a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8bfde648b604a2480767fe9b337d546)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:47.925176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48308913-8a36-4e3b-afc2-fd4a62e53a50 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:47.975404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-48308913-8a36-4e3b-afc2-fd4a62e53a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8bfde648b604a2480767fe9b337d546'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.975598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-48308913-8a36-4e3b-afc2-fd4a62e53a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8bfde648b604a2480767fe9b337d546'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.976253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-48308913-8a36-4e3b-afc2-fd4a62e53a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8bfde648b604a2480767fe9b337d546'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.985204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-48308913-8a36-4e3b-afc2-fd4a62e53a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8bfde648b604a2480767fe9b337d546'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:47.992968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-48308913-8a36-4e3b-afc2-fd4a62e53a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8bfde648b604a2480767fe9b337d546'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.012134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1179/2344] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:47 2022] DELETE /identity/v3/users/e8bfde648b604a2480767fe9b337d546 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:48.194565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:48.197457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:48.228923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177709a-5d50-4073-87d0-795b14e81b6c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:48.291233 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:48.292460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:48.292734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:48.293000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] PATH_INFO: `/v3/projects/e442497bec93476597f27e3478edf460` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:48.294105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e442497bec93476597f27e3478edf460)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:48.296008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:48.347650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e442497bec93476597f27e3478edf460', 'request_id': 'req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.354083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e442497bec93476597f27e3478edf460', 'request_id': 'req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.360777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e442497bec93476597f27e3478edf460', 'request_id': 'req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.374371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] The token cache is being invalidate because project e442497bec93476597f27e3478edf460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:48.374664 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.375291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bf3825da-a491-4c38-9ee2-73ec2c9d6c03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.376834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1166/2345] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:48 2022] DELETE /identity/v3/projects/e442497bec93476597f27e3478edf460 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:48.381947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a3421-f98a-481e-8ea8-fcc2e198ffa9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:48.382163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a3421-f98a-481e-8ea8-fcc2e198ffa9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:48.382439 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a3421-f98a-481e-8ea8-fcc2e198ffa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:48.405226 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-104a3421-f98a-481e-8ea8-fcc2e198ffa9 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:48.406435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-104a3421-f98a-481e-8ea8-fcc2e198ffa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:48.469845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1180/2346] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:48.478953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db781e-e0e0-4f79-ae9a-fe127b975b56 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:48.479084 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db781e-e0e0-4f79-ae9a-fe127b975b56 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:48.479399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db781e-e0e0-4f79-ae9a-fe127b975b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:48.499759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e3db781e-e0e0-4f79-ae9a-fe127b975b56 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:48.501014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e3db781e-e0e0-4f79-ae9a-fe127b975b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:48.509111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1167/2347] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:48.513977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a3421-f98a-481e-8ea8-fcc2e198ffa9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:48.523864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a3421-f98a-481e-8ea8-fcc2e198ffa9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:48.529067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb769c0d-8b82-485b-96fe-70f176b282c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:48.529672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb769c0d-8b82-485b-96fe-70f176b282c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:48.529953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb769c0d-8b82-485b-96fe-70f176b282c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:48.530236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb769c0d-8b82-485b-96fe-70f176b282c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:48.530873 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb769c0d-8b82-485b-96fe-70f176b282c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:48.531173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb769c0d-8b82-485b-96fe-70f176b282c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:48.532822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb769c0d-8b82-485b-96fe-70f176b282c8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:48.544798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1181/2348] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:48.549320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db781e-e0e0-4f79-ae9a-fe127b975b56 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:48.564656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db781e-e0e0-4f79-ae9a-fe127b975b56 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:48.570862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caaa11-ffcb-4a4a-805b-6de2f15a8533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:48.571471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caaa11-ffcb-4a4a-805b-6de2f15a8533 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:48.571783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caaa11-ffcb-4a4a-805b-6de2f15a8533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:48.572081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caaa11-ffcb-4a4a-805b-6de2f15a8533 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:48.573275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caaa11-ffcb-4a4a-805b-6de2f15a8533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:48.575303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caaa11-ffcb-4a4a-805b-6de2f15a8533 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:48.607610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1168/2349] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:48 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:48.609284 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb769c0d-8b82-485b-96fe-70f176b282c8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:48.619486 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb769c0d-8b82-485b-96fe-70f176b282c8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:48.627228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:48.681280 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef68118-f68b-4fab-84f4-24ed33f015fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:48.682026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef68118-f68b-4fab-84f4-24ed33f015fd admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:48.682451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef68118-f68b-4fab-84f4-24ed33f015fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:48.682736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef68118-f68b-4fab-84f4-24ed33f015fd admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:48.683371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef68118-f68b-4fab-84f4-24ed33f015fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:48.685525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef68118-f68b-4fab-84f4-24ed33f015fd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:48.711998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1182/2350] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:48 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:48.716699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caaa11-ffcb-4a4a-805b-6de2f15a8533 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:48.729535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caaa11-ffcb-4a4a-805b-6de2f15a8533 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:48.734871 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0c944-2d9f-45d5-a420-3722bf1706f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:48.735478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0c944-2d9f-45d5-a420-3722bf1706f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:48.735893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0c944-2d9f-45d5-a420-3722bf1706f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:48.736205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0c944-2d9f-45d5-a420-3722bf1706f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:48.737047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0c944-2d9f-45d5-a420-3722bf1706f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:48.738993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0c944-2d9f-45d5-a420-3722bf1706f4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:48.755866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1169/2351] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:48.764445 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef68118-f68b-4fab-84f4-24ed33f015fd admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:48.801667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef68118-f68b-4fab-84f4-24ed33f015fd admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:48.822695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a180c-f4e9-46a4-8af1-a10c04044437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:48.822695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a180c-f4e9-46a4-8af1-a10c04044437 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:48.822695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a180c-f4e9-46a4-8af1-a10c04044437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:48.822695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a180c-f4e9-46a4-8af1-a10c04044437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:48.823789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a180c-f4e9-46a4-8af1-a10c04044437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:48.823789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a180c-f4e9-46a4-8af1-a10c04044437 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:48.829365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1183/2352] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:48.833194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0c944-2d9f-45d5-a420-3722bf1706f4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:48.847037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0c944-2d9f-45d5-a420-3722bf1706f4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:48.858658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c9760-b88c-45ff-a61a-b936542ff0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:48.859415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c9760-b88c-45ff-a61a-b936542ff0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:48.859820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c9760-b88c-45ff-a61a-b936542ff0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:48.860193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c9760-b88c-45ff-a61a-b936542ff0f4 admin admin] PATH_INFO: `/v3/projects/b3d697163a8d4ee197f1b6b2856f08ae/users/2063027ade874ab19c7df1655ebd6b85/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:48.884322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c9760-b88c-45ff-a61a-b936542ff0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3d697163a8d4ee197f1b6b2856f08ae, user_id=2063027ade874ab19c7df1655ebd6b85, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:48.887936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c9760-b88c-45ff-a61a-b936542ff0f4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:48.910107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1170/2353] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:48 2022] PUT /identity/v3/projects/b3d697163a8d4ee197f1b6b2856f08ae/users/2063027ade874ab19c7df1655ebd6b85/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:48.920986 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a5565-fdd2-45de-8f4c-aa7e0d8499f0 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:48.921809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a5565-fdd2-45de-8f4c-aa7e0d8499f0 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:48.922422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a5565-fdd2-45de-8f4c-aa7e0d8499f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:48.957942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e96a5565-fdd2-45de-8f4c-aa7e0d8499f0 None None] MFA Rules not processed for user `2063027ade874ab19c7df1655ebd6b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:48.959932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e96a5565-fdd2-45de-8f4c-aa7e0d8499f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:49.041540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1184/2354] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:49.052851 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c9760-b88c-45ff-a61a-b936542ff0f4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:49.054595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-023c9760-b88c-45ff-a61a-b936542ff0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:49.075303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:49.115033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c9760-b88c-45ff-a61a-b936542ff0f4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:49.118678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a93056-48f4-4068-86cf-59b16fd22f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:49.119275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93056-48f4-4068-86cf-59b16fd22f4a service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:49.120267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93056-48f4-4068-86cf-59b16fd22f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:49.120548 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93056-48f4-4068-86cf-59b16fd22f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:49.128844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a93056-48f4-4068-86cf-59b16fd22f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:49.130256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a93056-48f4-4068-86cf-59b16fd22f4a service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:49.139110 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1171/2355] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:51.746708 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a537c88-ecf2-46ac-a144-e740a4d39706 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:51.747202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a537c88-ecf2-46ac-a144-e740a4d39706 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:51.747202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a537c88-ecf2-46ac-a144-e740a4d39706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:51.766964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6a537c88-ecf2-46ac-a144-e740a4d39706 None None] MFA Rules not processed for user `2063027ade874ab19c7df1655ebd6b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:51.768193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6a537c88-ecf2-46ac-a144-e740a4d39706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:51.775165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1185/2356] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:54.833337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a93056-48f4-4068-86cf-59b16fd22f4a service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:54.835723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-55a93056-48f4-4068-86cf-59b16fd22f4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:54.947926 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a93056-48f4-4068-86cf-59b16fd22f4a service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:54.953795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54221d6-c6dd-4a5e-84b9-308a547a8545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:54.954949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54221d6-c6dd-4a5e-84b9-308a547a8545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:54.955631 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54221d6-c6dd-4a5e-84b9-308a547a8545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:54.956154 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54221d6-c6dd-4a5e-84b9-308a547a8545 admin admin] PATH_INFO: `/v3/users/952e091d7ce94adab3af672e010b6259` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:54.957840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54221d6-c6dd-4a5e-84b9-308a547a8545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=952e091d7ce94adab3af672e010b6259)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:54.960376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54221d6-c6dd-4a5e-84b9-308a547a8545 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:55.061754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a54221d6-c6dd-4a5e-84b9-308a547a8545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '952e091d7ce94adab3af672e010b6259'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:55.061754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a54221d6-c6dd-4a5e-84b9-308a547a8545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '952e091d7ce94adab3af672e010b6259'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:55.062644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a54221d6-c6dd-4a5e-84b9-308a547a8545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '952e091d7ce94adab3af672e010b6259'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:55.079135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a54221d6-c6dd-4a5e-84b9-308a547a8545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '952e091d7ce94adab3af672e010b6259'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:55.119045 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a54221d6-c6dd-4a5e-84b9-308a547a8545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '952e091d7ce94adab3af672e010b6259'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:55.145780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c22b0-adb3-404c-bb23-49b37a1962f2 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:55.146632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c22b0-adb3-404c-bb23-49b37a1962f2 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:55.146934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c22b0-adb3-404c-bb23-49b37a1962f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:55.155889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1172/2357] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:54 2022] DELETE /identity/v3/users/952e091d7ce94adab3af672e010b6259 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:55.158219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717c3ce-d91e-4fd8-b906-9963e3861137 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:55.158515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717c3ce-d91e-4fd8-b906-9963e3861137 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:55.158792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717c3ce-d91e-4fd8-b906-9963e3861137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:55.168356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e97c22b0-adb3-404c-bb23-49b37a1962f2 None None] MFA Rules not processed for user `731cf6df5bda42148e1420e365306445`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:55.169574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e97c22b0-adb3-404c-bb23-49b37a1962f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:55.177850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a717c3ce-d91e-4fd8-b906-9963e3861137 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:55.179333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a717c3ce-d91e-4fd8-b906-9963e3861137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:55.218382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:55.230576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1186/2358] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:55.250371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1173/2359] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:55.261711 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c22b0-adb3-404c-bb23-49b37a1962f2 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:55.263152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e97c22b0-adb3-404c-bb23-49b37a1962f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:55.288261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:55.348981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c22b0-adb3-404c-bb23-49b37a1962f2 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:55.353035 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b690e96-3282-4495-806f-cddcfeeced0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:55.353646 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b690e96-3282-4495-806f-cddcfeeced0d service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:55.353927 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b690e96-3282-4495-806f-cddcfeeced0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:55.354198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b690e96-3282-4495-806f-cddcfeeced0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:55.368089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b690e96-3282-4495-806f-cddcfeeced0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:55.369562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b690e96-3282-4495-806f-cddcfeeced0d service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:55.380875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1187/2360] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:04:55.905626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717c3ce-d91e-4fd8-b906-9963e3861137 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:55.908539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a717c3ce-d91e-4fd8-b906-9963e3861137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:55.923836 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717c3ce-d91e-4fd8-b906-9963e3861137 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:55.929656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:55.930662 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:55.931161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:55.931782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] PATH_INFO: `/v3/projects/b862d2489d67446ab37c5f9d4b3322d2` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:55.933794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b862d2489d67446ab37c5f9d4b3322d2)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:55.943325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.020537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b862d2489d67446ab37c5f9d4b3322d2', 'request_id': 'req-5011f841-1c8b-4df3-a2d1-55404feca4ec'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:56.031574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b862d2489d67446ab37c5f9d4b3322d2', 'request_id': 'req-5011f841-1c8b-4df3-a2d1-55404feca4ec'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:56.039509 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b862d2489d67446ab37c5f9d4b3322d2', 'request_id': 'req-5011f841-1c8b-4df3-a2d1-55404feca4ec'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:56.055961 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] The token cache is being invalidate because project b862d2489d67446ab37c5f9d4b3322d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:56.056388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:56.057155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5011f841-1c8b-4df3-a2d1-55404feca4ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:56.060151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1174/2361] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:55 2022] DELETE /identity/v3/projects/b862d2489d67446ab37c5f9d4b3322d2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:56.064602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a792f8-43b6-4c35-b181-8edb596a7281 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.064899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a792f8-43b6-4c35-b181-8edb596a7281 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.065180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a792f8-43b6-4c35-b181-8edb596a7281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.110816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e5a792f8-43b6-4c35-b181-8edb596a7281 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:56.112541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e5a792f8-43b6-4c35-b181-8edb596a7281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.200921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1188/2362] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:56.209382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66833ab6-aab2-48ea-b755-5647ffc78f84 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.209711 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66833ab6-aab2-48ea-b755-5647ffc78f84 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.209948 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66833ab6-aab2-48ea-b755-5647ffc78f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.246281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-66833ab6-aab2-48ea-b755-5647ffc78f84 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:56.248445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-66833ab6-aab2-48ea-b755-5647ffc78f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.261043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1175/2363] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:56.265206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a792f8-43b6-4c35-b181-8edb596a7281 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:56.277266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a792f8-43b6-4c35-b181-8edb596a7281 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:56.283157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd85796-0ad9-4343-a637-855908b9c499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:56.283868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd85796-0ad9-4343-a637-855908b9c499 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.284242 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd85796-0ad9-4343-a637-855908b9c499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.284663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd85796-0ad9-4343-a637-855908b9c499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.285680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd85796-0ad9-4343-a637-855908b9c499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:56.286058 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd85796-0ad9-4343-a637-855908b9c499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:56.287894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd85796-0ad9-4343-a637-855908b9c499 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.302624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1189/2364] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:04:56.307936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66833ab6-aab2-48ea-b755-5647ffc78f84 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:56.318929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66833ab6-aab2-48ea-b755-5647ffc78f84 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:56.327194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceac65d-cd24-43c1-ad42-e050e37a10eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:56.327924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac65d-cd24-43c1-ad42-e050e37a10eb admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.328356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac65d-cd24-43c1-ad42-e050e37a10eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.328771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac65d-cd24-43c1-ad42-e050e37a10eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.329919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceac65d-cd24-43c1-ad42-e050e37a10eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:56.333035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceac65d-cd24-43c1-ad42-e050e37a10eb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.376984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1176/2365] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:56 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:56.378431 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd85796-0ad9-4343-a637-855908b9c499 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:56.392235 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd85796-0ad9-4343-a637-855908b9c499 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:56.474309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e4df8-dfce-478d-a51e-fed847b598a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:56.474994 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e4df8-dfce-478d-a51e-fed847b598a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.475336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e4df8-dfce-478d-a51e-fed847b598a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.475677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e4df8-dfce-478d-a51e-fed847b598a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.476411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e4df8-dfce-478d-a51e-fed847b598a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:56.478386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e4df8-dfce-478d-a51e-fed847b598a3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.501451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1190/2366] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:56 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:56.505547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceac65d-cd24-43c1-ad42-e050e37a10eb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:56.517063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceac65d-cd24-43c1-ad42-e050e37a10eb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:56.523061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fd179-2928-44bb-be63-dfa3e16f368b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:56.523668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fd179-2928-44bb-be63-dfa3e16f368b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.523963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fd179-2928-44bb-be63-dfa3e16f368b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.524253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fd179-2928-44bb-be63-dfa3e16f368b admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.524935 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fd179-2928-44bb-be63-dfa3e16f368b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:56.527016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fd179-2928-44bb-be63-dfa3e16f368b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.541346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1177/2367] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:56.543131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e4df8-dfce-478d-a51e-fed847b598a3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:56.553192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e4df8-dfce-478d-a51e-fed847b598a3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:56.558169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2e929-ce38-4988-b658-1afb52e3d930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:56.558938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2e929-ce38-4988-b658-1afb52e3d930 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.559251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2e929-ce38-4988-b658-1afb52e3d930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.559521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2e929-ce38-4988-b658-1afb52e3d930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.560201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2e929-ce38-4988-b658-1afb52e3d930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:56.561808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2e929-ce38-4988-b658-1afb52e3d930 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.572889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1191/2368] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:56.577831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fd179-2928-44bb-be63-dfa3e16f368b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:56.588206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fd179-2928-44bb-be63-dfa3e16f368b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:56.593511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a39861-75b7-43e4-ac62-89a5293a5e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:56.594238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a39861-75b7-43e4-ac62-89a5293a5e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.594584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a39861-75b7-43e4-ac62-89a5293a5e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.594927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a39861-75b7-43e4-ac62-89a5293a5e2e admin admin] PATH_INFO: `/v3/projects/dc33f347b062491aaa04b7dc18553cf4/users/3ccfd2f7e35646b0a60b28b6807cee4b/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.610035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a39861-75b7-43e4-ac62-89a5293a5e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc33f347b062491aaa04b7dc18553cf4, user_id=3ccfd2f7e35646b0a60b28b6807cee4b, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:56.612734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a39861-75b7-43e4-ac62-89a5293a5e2e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.625898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1178/2369] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:56 2022] PUT /identity/v3/projects/dc33f347b062491aaa04b7dc18553cf4/users/3ccfd2f7e35646b0a60b28b6807cee4b/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:56.630796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690c3c3-edd0-493b-9579-e597b043de12 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.631093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690c3c3-edd0-493b-9579-e597b043de12 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.631368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690c3c3-edd0-493b-9579-e597b043de12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.650888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3690c3c3-edd0-493b-9579-e597b043de12 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:56.652813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3690c3c3-edd0-493b-9579-e597b043de12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.673901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:56.715211 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1192/2370] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:56.725055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a39861-75b7-43e4-ac62-89a5293a5e2e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:56.726944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f6a39861-75b7-43e4-ac62-89a5293a5e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.807020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a39861-75b7-43e4-ac62-89a5293a5e2e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:56.811049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c9fe3-618e-4f37-9404-7dc64172dc1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:56.811660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9fe3-618e-4f37-9404-7dc64172dc1e service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.811941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9fe3-618e-4f37-9404-7dc64172dc1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.812243 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9fe3-618e-4f37-9404-7dc64172dc1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.821878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c9fe3-618e-4f37-9404-7dc64172dc1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:56.823832 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c9fe3-618e-4f37-9404-7dc64172dc1e service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.828739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:56.834845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1179/2371] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:03.798813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690c3c3-edd0-493b-9579-e597b043de12 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:03.800595 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3690c3c3-edd0-493b-9579-e597b043de12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:03.814986 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690c3c3-edd0-493b-9579-e597b043de12 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:03.831962 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:03.832935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:03.833398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:03.833837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] PATH_INFO: `/v3/users/86639ca833cd4fac83c404768dc7ae92` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:03.837543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86639ca833cd4fac83c404768dc7ae92)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:03.840921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:03.906679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86639ca833cd4fac83c404768dc7ae92'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:03.907411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86639ca833cd4fac83c404768dc7ae92'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:03.911033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86639ca833cd4fac83c404768dc7ae92'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:03.923573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86639ca833cd4fac83c404768dc7ae92'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:03.939808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86639ca833cd4fac83c404768dc7ae92'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:03.968088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1193/2372] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:05:03 2022] DELETE /identity/v3/users/86639ca833cd4fac83c404768dc7ae92 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:05:03.972433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00159b-02d4-4a99-92a8-9057f951003a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:03.972940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00159b-02d4-4a99-92a8-9057f951003a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:03.973043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00159b-02d4-4a99-92a8-9057f951003a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:04.001236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5e00159b-02d4-4a99-92a8-9057f951003a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:04.003411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5e00159b-02d4-4a99-92a8-9057f951003a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:04.097968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1180/2373] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:04.125169 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:04.126852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:04.230036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea9f2d-c55c-4b8b-9f01-9be2e42ffba2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:04.243968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8d1f4-fbf4-49c0-be5f-e38d62162dbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:04.245440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8d1f4-fbf4-49c0-be5f-e38d62162dbe service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:04.245779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8d1f4-fbf4-49c0-be5f-e38d62162dbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:04.247421 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8d1f4-fbf4-49c0-be5f-e38d62162dbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:04.273079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8d1f4-fbf4-49c0-be5f-e38d62162dbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:04.274573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8d1f4-fbf4-49c0-be5f-e38d62162dbe service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:04.301554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1194/2374] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:04.845783 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eece79-22f6-42b1-9185-c5b1ac175752 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:04.846229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eece79-22f6-42b1-9185-c5b1ac175752 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:04.846660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eece79-22f6-42b1-9185-c5b1ac175752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:04.903811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a8eece79-22f6-42b1-9185-c5b1ac175752 None None] MFA Rules not processed for user `3ccfd2f7e35646b0a60b28b6807cee4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:04.905340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a8eece79-22f6-42b1-9185-c5b1ac175752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:05.002997 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1181/2375] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:05:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:05.007395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8d1f4-fbf4-49c0-be5f-e38d62162dbe service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:05.008620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9cd8d1f4-fbf4-49c0-be5f-e38d62162dbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:05.033150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8d1f4-fbf4-49c0-be5f-e38d62162dbe service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:05.044611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826da786-53d6-48f1-b821-87295fafbaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:05.045242 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826da786-53d6-48f1-b821-87295fafbaf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:05.045655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826da786-53d6-48f1-b821-87295fafbaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:05.046019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826da786-53d6-48f1-b821-87295fafbaf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:05.046438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826da786-53d6-48f1-b821-87295fafbaf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:05.048517 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826da786-53d6-48f1-b821-87295fafbaf1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:05.101176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1195/2376] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:05:05 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:05:05.104979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eece79-22f6-42b1-9185-c5b1ac175752 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:05.118083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eece79-22f6-42b1-9185-c5b1ac175752 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:05.189528 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bc942-b6e7-4318-8aae-39814844c5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:05.190270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bc942-b6e7-4318-8aae-39814844c5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:05.190555 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bc942-b6e7-4318-8aae-39814844c5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:05.191052 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bc942-b6e7-4318-8aae-39814844c5a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:05.191963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987bc942-b6e7-4318-8aae-39814844c5a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:05.193983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987bc942-b6e7-4318-8aae-39814844c5a9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:05.224647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1182/2377] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:05:05 2022] POST /identity/v3/users => generated 295 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:05:05.229308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826da786-53d6-48f1-b821-87295fafbaf1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:05.240173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826da786-53d6-48f1-b821-87295fafbaf1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:05.246755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535734a-c728-4868-8bb3-747a9960c921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:05.247655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535734a-c728-4868-8bb3-747a9960c921 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:05.248151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535734a-c728-4868-8bb3-747a9960c921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:05.248596 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535734a-c728-4868-8bb3-747a9960c921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:05.249422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535734a-c728-4868-8bb3-747a9960c921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:05.251228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535734a-c728-4868-8bb3-747a9960c921 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:05.265047 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1196/2378] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:05:05.269418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bc942-b6e7-4318-8aae-39814844c5a9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:05.280413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bc942-b6e7-4318-8aae-39814844c5a9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:05.286338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fbdc6-ce2e-48b3-8d4c-4e133190c00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:05.287266 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fbdc6-ce2e-48b3-8d4c-4e133190c00d admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:05.287712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fbdc6-ce2e-48b3-8d4c-4e133190c00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:05.288198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fbdc6-ce2e-48b3-8d4c-4e133190c00d admin admin] PATH_INFO: `/v3/projects/ad806acbc6e84af6a46ddf2c13eb798f/users/147ad0a9182d40128b64ecda50abd8a0/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:05.306211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fbdc6-ce2e-48b3-8d4c-4e133190c00d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad806acbc6e84af6a46ddf2c13eb798f, user_id=147ad0a9182d40128b64ecda50abd8a0, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:05.309982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fbdc6-ce2e-48b3-8d4c-4e133190c00d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:05.314021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:05.336818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1183/2379] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:05:05 2022] PUT /identity/v3/projects/ad806acbc6e84af6a46ddf2c13eb798f/users/147ad0a9182d40128b64ecda50abd8a0/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:05:05.340383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535734a-c728-4868-8bb3-747a9960c921 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:05.353936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535734a-c728-4868-8bb3-747a9960c921 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:05.360643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:05.485538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cbdb9-8e17-43a0-983f-6fa8dd93de8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:05.486530 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cbdb9-8e17-43a0-983f-6fa8dd93de8e admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:05.488101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cbdb9-8e17-43a0-983f-6fa8dd93de8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:05.488456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cbdb9-8e17-43a0-983f-6fa8dd93de8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:05.489480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cbdb9-8e17-43a0-983f-6fa8dd93de8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:05.491548 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cbdb9-8e17-43a0-983f-6fa8dd93de8e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:05.497071 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fbdc6-ce2e-48b3-8d4c-4e133190c00d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:05.499419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7b3fbdc6-ce2e-48b3-8d4c-4e133190c00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:05.508909 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1197/2380] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:05:05.513753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cbdb9-8e17-43a0-983f-6fa8dd93de8e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:05.542632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cbdb9-8e17-43a0-983f-6fa8dd93de8e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:05.546605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fbdc6-ce2e-48b3-8d4c-4e133190c00d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:05.558127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57492d19-051a-49c4-a808-413613048b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:05.558957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57492d19-051a-49c4-a808-413613048b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:05.560356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57492d19-051a-49c4-a808-413613048b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:05.561939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57492d19-051a-49c4-a808-413613048b54 admin admin] PATH_INFO: `/v3/projects/ad806acbc6e84af6a46ddf2c13eb798f/users/147ad0a9182d40128b64ecda50abd8a0/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:05.569557 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57492d19-051a-49c4-a808-413613048b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad806acbc6e84af6a46ddf2c13eb798f, user_id=147ad0a9182d40128b64ecda50abd8a0, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:05.571306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:05.572035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:05.572991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:05.573471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] PATH_INFO: `/v3/projects/93d64e0a3ca74d278f80c7b680055baa` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:05.577733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57492d19-051a-49c4-a808-413613048b54 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:05.578072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=93d64e0a3ca74d278f80c7b680055baa)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:05.580010 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:05.602994 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1198/2381] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:05:05 2022] PUT /identity/v3/projects/ad806acbc6e84af6a46ddf2c13eb798f/users/147ad0a9182d40128b64ecda50abd8a0/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:05:05.690785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93d64e0a3ca74d278f80c7b680055baa', 'request_id': 'req-6b10e140-a01e-46a2-9143-93d392ef6d0a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:05.708241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93d64e0a3ca74d278f80c7b680055baa', 'request_id': 'req-6b10e140-a01e-46a2-9143-93d392ef6d0a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:05.722455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93d64e0a3ca74d278f80c7b680055baa', 'request_id': 'req-6b10e140-a01e-46a2-9143-93d392ef6d0a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:05.743026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] The token cache is being invalidate because project 93d64e0a3ca74d278f80c7b680055baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:05:05.743473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:05.744912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6b10e140-a01e-46a2-9143-93d392ef6d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:05.746786 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1184/2382] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:05:05 2022] DELETE /identity/v3/projects/93d64e0a3ca74d278f80c7b680055baa => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:05:05.753378 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3607e-480a-4f3b-9edf-00e733fb068f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:05.753659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3607e-480a-4f3b-9edf-00e733fb068f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:05.753917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3607e-480a-4f3b-9edf-00e733fb068f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:05.789047 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-23e3607e-480a-4f3b-9edf-00e733fb068f None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:05.790495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-23e3607e-480a-4f3b-9edf-00e733fb068f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:05.980803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1199/2383] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:05.989192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee7b4e-7a41-49a4-9bb4-02a936478f91 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:05.989569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee7b4e-7a41-49a4-9bb4-02a936478f91 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:05.990042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee7b4e-7a41-49a4-9bb4-02a936478f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:06.025785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-24ee7b4e-7a41-49a4-9bb4-02a936478f91 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:06.027016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-24ee7b4e-7a41-49a4-9bb4-02a936478f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:06.064314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1185/2384] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:06.069228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3607e-480a-4f3b-9edf-00e733fb068f None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:06.085196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3607e-480a-4f3b-9edf-00e733fb068f None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:06.095724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce149c3-a00b-4828-84d5-c695c48bad4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:06.096643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce149c3-a00b-4828-84d5-c695c48bad4a admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:06.097106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce149c3-a00b-4828-84d5-c695c48bad4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:06.097569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce149c3-a00b-4828-84d5-c695c48bad4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:06.098578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce149c3-a00b-4828-84d5-c695c48bad4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:05:06.099239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce149c3-a00b-4828-84d5-c695c48bad4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:06.101683 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce149c3-a00b-4828-84d5-c695c48bad4a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:06.122051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1200/2385] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:05:06.126476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee7b4e-7a41-49a4-9bb4-02a936478f91 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:06.139418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee7b4e-7a41-49a4-9bb4-02a936478f91 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:06.153824 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb7621-f53c-4809-b64e-498fad21fb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:06.154781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb7621-f53c-4809-b64e-498fad21fb8a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:06.158411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb7621-f53c-4809-b64e-498fad21fb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:06.158881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb7621-f53c-4809-b64e-498fad21fb8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:06.159941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb7621-f53c-4809-b64e-498fad21fb8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:06.162093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb7621-f53c-4809-b64e-498fad21fb8a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:06.223308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1186/2386] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:05:06 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:05:06.226598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce149c3-a00b-4828-84d5-c695c48bad4a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:06.241057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce149c3-a00b-4828-84d5-c695c48bad4a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:06.426606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2aafed-c713-4258-b3d0-812d6a1c62e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:06.428070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aafed-c713-4258-b3d0-812d6a1c62e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:06.428359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aafed-c713-4258-b3d0-812d6a1c62e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:06.428634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aafed-c713-4258-b3d0-812d6a1c62e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:06.429280 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2aafed-c713-4258-b3d0-812d6a1c62e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:06.431451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2aafed-c713-4258-b3d0-812d6a1c62e6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:06.479383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1201/2387] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:05:06 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:05:06.480700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb7621-f53c-4809-b64e-498fad21fb8a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:06.509592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb7621-f53c-4809-b64e-498fad21fb8a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:06.525949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485634a-9953-45cc-9ec9-5e01bd70da4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:06.526699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485634a-9953-45cc-9ec9-5e01bd70da4f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:06.526978 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485634a-9953-45cc-9ec9-5e01bd70da4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:06.527253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485634a-9953-45cc-9ec9-5e01bd70da4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:06.527979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5485634a-9953-45cc-9ec9-5e01bd70da4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:06.532404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5485634a-9953-45cc-9ec9-5e01bd70da4f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:06.574128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1187/2388] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:05:06.579691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2aafed-c713-4258-b3d0-812d6a1c62e6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:06.598981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2aafed-c713-4258-b3d0-812d6a1c62e6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:06.623404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08bd82-9334-49fa-9f93-9ab00e91bddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:06.624746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08bd82-9334-49fa-9f93-9ab00e91bddc admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:06.625030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08bd82-9334-49fa-9f93-9ab00e91bddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:06.626318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08bd82-9334-49fa-9f93-9ab00e91bddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:06.627571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08bd82-9334-49fa-9f93-9ab00e91bddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:06.630374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08bd82-9334-49fa-9f93-9ab00e91bddc admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:06.644794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:06.679269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1202/2389] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:05:06.683830 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485634a-9953-45cc-9ec9-5e01bd70da4f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:06.705638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485634a-9953-45cc-9ec9-5e01bd70da4f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:06.728286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02282f6c-d289-4370-953c-faced83236ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:06.729672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282f6c-d289-4370-953c-faced83236ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:06.729961 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282f6c-d289-4370-953c-faced83236ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:06.730244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282f6c-d289-4370-953c-faced83236ee admin admin] PATH_INFO: `/v3/projects/5294ff8dc07444e5b8758dbae2db7fc4/users/bea2aecac254429a823b8765feb6c767/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:06.793368 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02282f6c-d289-4370-953c-faced83236ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5294ff8dc07444e5b8758dbae2db7fc4, user_id=bea2aecac254429a823b8765feb6c767, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:06.799093 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02282f6c-d289-4370-953c-faced83236ee admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:06.850582 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1188/2390] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:05:06 2022] PUT /identity/v3/projects/5294ff8dc07444e5b8758dbae2db7fc4/users/bea2aecac254429a823b8765feb6c767/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:05:06.853090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4d873-c38f-4e19-a495-c51eb18c4a1f None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:06.853382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4d873-c38f-4e19-a495-c51eb18c4a1f None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:06.853694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4d873-c38f-4e19-a495-c51eb18c4a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:06.900656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-14b4d873-c38f-4e19-a495-c51eb18c4a1f None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:06.903195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-14b4d873-c38f-4e19-a495-c51eb18c4a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:07.009555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1203/2391] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:07.066998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02282f6c-d289-4370-953c-faced83236ee admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:07.070026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-02282f6c-d289-4370-953c-faced83236ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:07.274408 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02282f6c-d289-4370-953c-faced83236ee admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:07.305549 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9ea03-c908-42c8-8808-fd987ed4836b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:07.306403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9ea03-c908-42c8-8808-fd987ed4836b service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:07.307387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9ea03-c908-42c8-8808-fd987ed4836b service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:07.307810 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9ea03-c908-42c8-8808-fd987ed4836b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:07.334661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9ea03-c908-42c8-8808-fd987ed4836b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:07.337106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9ea03-c908-42c8-8808-fd987ed4836b service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:07.387322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1189/2392] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:07.613054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d90d2-ee4f-421b-be80-2946de984ded None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:07.613528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d90d2-ee4f-421b-be80-2946de984ded None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:07.613624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d90d2-ee4f-421b-be80-2946de984ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:07.655996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-410d90d2-ee4f-421b-be80-2946de984ded None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:07.657213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-410d90d2-ee4f-421b-be80-2946de984ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:07.765174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1204/2393] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:05:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:05:07.825803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9ea03-c908-42c8-8808-fd987ed4836b service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:07.827802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ecd9ea03-c908-42c8-8808-fd987ed4836b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:07.862059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:07.985133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9ea03-c908-42c8-8808-fd987ed4836b service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:07.994860 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ee8dd-6533-4ef7-9646-7b51180a50c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:07.995974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ee8dd-6533-4ef7-9646-7b51180a50c7 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:07.997429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ee8dd-6533-4ef7-9646-7b51180a50c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:07.998449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ee8dd-6533-4ef7-9646-7b51180a50c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:08.025279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ee8dd-6533-4ef7-9646-7b51180a50c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:08.033487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ee8dd-6533-4ef7-9646-7b51180a50c7 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:08.088729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1190/2394] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:10.741525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947852c-f27b-49af-bfa2-6056215544ba None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:10.742151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947852c-f27b-49af-bfa2-6056215544ba None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:10.742151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947852c-f27b-49af-bfa2-6056215544ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:10.810549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-7947852c-f27b-49af-bfa2-6056215544ba None None] MFA Rules not processed for user `46c01f8b99e24be5b6c2d73b9b1bd9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:10.818604 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7947852c-f27b-49af-bfa2-6056215544ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:11.071182 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1205/2395] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:05:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:18.042917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=24 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:21.045302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=78 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:21.560818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe13799-5f29-4752-b460-0c5f597c1397 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:21.561356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe13799-5f29-4752-b460-0c5f597c1397 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:21.563286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe13799-5f29-4752-b460-0c5f597c1397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:21.626091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2fe13799-5f29-4752-b460-0c5f597c1397 None None] MFA Rules not processed for user `147ad0a9182d40128b64ecda50abd8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:21.627501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2fe13799-5f29-4752-b460-0c5f597c1397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:21.753109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1191/2396] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:05:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:21.976234 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947852c-f27b-49af-bfa2-6056215544ba None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:21.977723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-7947852c-f27b-49af-bfa2-6056215544ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:22.205238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947852c-f27b-49af-bfa2-6056215544ba None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:22.219516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21876e0a-3ae4-4ef5-b5f8-8a91e2520b16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:22.224440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21876e0a-3ae4-4ef5-b5f8-8a91e2520b16 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:22.224777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21876e0a-3ae4-4ef5-b5f8-8a91e2520b16 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:22.225056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21876e0a-3ae4-4ef5-b5f8-8a91e2520b16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:22.229043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-21876e0a-3ae4-4ef5-b5f8-8a91e2520b16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:22.379184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21876e0a-3ae4-4ef5-b5f8-8a91e2520b16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:22.381153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21876e0a-3ae4-4ef5-b5f8-8a91e2520b16 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:22.392023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1206/2397] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:26.479151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048136-6fdf-4a6e-80b5-a671e1daf819 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:26.479715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048136-6fdf-4a6e-80b5-a671e1daf819 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:26.479715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048136-6fdf-4a6e-80b5-a671e1daf819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:26.538730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d0048136-6fdf-4a6e-80b5-a671e1daf819 None None] MFA Rules not processed for user `bea2aecac254429a823b8765feb6c767`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:26.540715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d0048136-6fdf-4a6e-80b5-a671e1daf819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:26.669747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1192/2398] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:05:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:26.675290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21876e0a-3ae4-4ef5-b5f8-8a91e2520b16 service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:26.694914 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21876e0a-3ae4-4ef5-b5f8-8a91e2520b16 service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:26.707837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e7537-a387-4c1c-9f3d-0acfe6be44a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:26.711029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e7537-a387-4c1c-9f3d-0acfe6be44a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:26.712421 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e7537-a387-4c1c-9f3d-0acfe6be44a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:26.712852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e7537-a387-4c1c-9f3d-0acfe6be44a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:26.713728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e7537-a387-4c1c-9f3d-0acfe6be44a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:26.723427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e7537-a387-4c1c-9f3d-0acfe6be44a8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:26.803304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1207/2399] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:05:26 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:05:26.804369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0048136-6fdf-4a6e-80b5-a671e1daf819 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:26.827123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0048136-6fdf-4a6e-80b5-a671e1daf819 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:27.048434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c47a2-75ac-4175-838a-0ed315e9654f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:27.049091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c47a2-75ac-4175-838a-0ed315e9654f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:27.049382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c47a2-75ac-4175-838a-0ed315e9654f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:27.049656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c47a2-75ac-4175-838a-0ed315e9654f admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:27.050305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c47a2-75ac-4175-838a-0ed315e9654f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:27.054139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c47a2-75ac-4175-838a-0ed315e9654f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:27.114281 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1193/2400] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:05:26 2022] POST /identity/v3/users => generated 297 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:05:27.115477 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e7537-a387-4c1c-9f3d-0acfe6be44a8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:27.128148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e7537-a387-4c1c-9f3d-0acfe6be44a8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:27.149907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3079c-fb41-4656-9cf8-bdf9c0b62c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:27.158432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3079c-fb41-4656-9cf8-bdf9c0b62c50 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:27.159342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3079c-fb41-4656-9cf8-bdf9c0b62c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:27.161531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3079c-fb41-4656-9cf8-bdf9c0b62c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:27.172611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3079c-fb41-4656-9cf8-bdf9c0b62c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:27.177528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3079c-fb41-4656-9cf8-bdf9c0b62c50 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:27.238921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c47a2-75ac-4175-838a-0ed315e9654f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:27.249596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:27.256367 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c47a2-75ac-4175-838a-0ed315e9654f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:27.257371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1208/2401] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:05:27.275853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d3c1b-0e49-4a69-b1d8-8c09adc32392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:27.276510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d3c1b-0e49-4a69-b1d8-8c09adc32392 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:27.276796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d3c1b-0e49-4a69-b1d8-8c09adc32392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:27.277079 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d3c1b-0e49-4a69-b1d8-8c09adc32392 admin admin] PATH_INFO: `/v3/projects/486f37b1a50145a3a8b9deac00abb96a/users/3a35aea9bd23453dbca1a83a9513dd60/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:27.303357 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d3c1b-0e49-4a69-b1d8-8c09adc32392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486f37b1a50145a3a8b9deac00abb96a, user_id=3a35aea9bd23453dbca1a83a9513dd60, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:27.306956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d3c1b-0e49-4a69-b1d8-8c09adc32392 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:27.340469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1194/2402] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:05:27 2022] PUT /identity/v3/projects/486f37b1a50145a3a8b9deac00abb96a/users/3a35aea9bd23453dbca1a83a9513dd60/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:05:27.344304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3079c-fb41-4656-9cf8-bdf9c0b62c50 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:27.355148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3079c-fb41-4656-9cf8-bdf9c0b62c50 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:27.504996 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe0ad8-cda2-450d-9510-e6e77ff64153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:27.505636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe0ad8-cda2-450d-9510-e6e77ff64153 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:27.505934 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe0ad8-cda2-450d-9510-e6e77ff64153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:27.506212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe0ad8-cda2-450d-9510-e6e77ff64153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:27.506875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe0ad8-cda2-450d-9510-e6e77ff64153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:27.509090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe0ad8-cda2-450d-9510-e6e77ff64153 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:27.519299 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:27.539122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1209/2403] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:05:27.541004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d3c1b-0e49-4a69-b1d8-8c09adc32392 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:27.552698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d3c1b-0e49-4a69-b1d8-8c09adc32392 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:27.598411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a2831-39ce-44d0-af8f-e23b2bb4f586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:27.599162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a2831-39ce-44d0-af8f-e23b2bb4f586 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:27.599538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a2831-39ce-44d0-af8f-e23b2bb4f586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:27.599846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a2831-39ce-44d0-af8f-e23b2bb4f586 admin admin] PATH_INFO: `/v3/projects/486f37b1a50145a3a8b9deac00abb96a/users/3a35aea9bd23453dbca1a83a9513dd60/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:27.615505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a2831-39ce-44d0-af8f-e23b2bb4f586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486f37b1a50145a3a8b9deac00abb96a, user_id=3a35aea9bd23453dbca1a83a9513dd60, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:27.618380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a2831-39ce-44d0-af8f-e23b2bb4f586 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:27.637239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1195/2404] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:05:27 2022] PUT /identity/v3/projects/486f37b1a50145a3a8b9deac00abb96a/users/3a35aea9bd23453dbca1a83a9513dd60/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:05:32.926583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7c0f3-3dbe-416a-9c8f-f55a14e00556 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:32.926583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7c0f3-3dbe-416a-9c8f-f55a14e00556 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:32.926583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7c0f3-3dbe-416a-9c8f-f55a14e00556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:32.960761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3ff7c0f3-3dbe-416a-9c8f-f55a14e00556 None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:32.965261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3ff7c0f3-3dbe-416a-9c8f-f55a14e00556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:33.105561 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1210/2405] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:05:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:33.115611 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a2831-39ce-44d0-af8f-e23b2bb4f586 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:33.324694 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a2831-39ce-44d0-af8f-e23b2bb4f586 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:33.332753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63694e-c814-467b-ac05-2a3821fa0a27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:33.334918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63694e-c814-467b-ac05-2a3821fa0a27 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:33.335411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63694e-c814-467b-ac05-2a3821fa0a27 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:33.335944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63694e-c814-467b-ac05-2a3821fa0a27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:33.348987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63694e-c814-467b-ac05-2a3821fa0a27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:33.354133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63694e-c814-467b-ac05-2a3821fa0a27 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:33.378893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1196/2406] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:34.561948 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7c0f3-3dbe-416a-9c8f-f55a14e00556 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:34.564406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3ff7c0f3-3dbe-416a-9c8f-f55a14e00556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:34.736456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7c0f3-3dbe-416a-9c8f-f55a14e00556 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:34.741048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f219d-a1c6-490a-92ea-91f9eb9ff3de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:34.741882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f219d-a1c6-490a-92ea-91f9eb9ff3de service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:34.742255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f219d-a1c6-490a-92ea-91f9eb9ff3de service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:34.742515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f219d-a1c6-490a-92ea-91f9eb9ff3de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:34.744416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-579f219d-a1c6-490a-92ea-91f9eb9ff3de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:34.896413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f219d-a1c6-490a-92ea-91f9eb9ff3de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:34.900273 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f219d-a1c6-490a-92ea-91f9eb9ff3de service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:34.917701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1211/2407] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:36.919928 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d77de-91d1-4be2-a429-bfdc9ccd01be None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:36.920454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d77de-91d1-4be2-a429-bfdc9ccd01be None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:36.920556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d77de-91d1-4be2-a429-bfdc9ccd01be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:36.983844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-534d77de-91d1-4be2-a429-bfdc9ccd01be None None] MFA Rules not processed for user `3a35aea9bd23453dbca1a83a9513dd60`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:36.985794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-534d77de-91d1-4be2-a429-bfdc9ccd01be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:37.130381 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1197/2408] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:05:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:42.009334 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f219d-a1c6-490a-92ea-91f9eb9ff3de service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:42.017681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-579f219d-a1c6-490a-92ea-91f9eb9ff3de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:42.161134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f219d-a1c6-490a-92ea-91f9eb9ff3de service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:42.166588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbe293-9f00-4706-bbe7-4efd436a6b58 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '794dbf38682e4b4f939854eecbacaf07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:42.166588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbe293-9f00-4706-bbe7-4efd436a6b58 service glance] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:42.166588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbe293-9f00-4706-bbe7-4efd436a6b58 service glance] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:42.176854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbe293-9f00-4706-bbe7-4efd436a6b58 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:42.209201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:42.350077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbe293-9f00-4706-bbe7-4efd436a6b58 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:42.351590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbe293-9f00-4706-bbe7-4efd436a6b58 service glance] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:42.382604 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1212/2409] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:42.650002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230479c3-b0bc-48ef-b4a6-50d4028c635f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:42.650002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230479c3-b0bc-48ef-b4a6-50d4028c635f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:42.650002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230479c3-b0bc-48ef-b4a6-50d4028c635f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:42.680834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-230479c3-b0bc-48ef-b4a6-50d4028c635f None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:42.683273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-230479c3-b0bc-48ef-b4a6-50d4028c635f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:42.695669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1198/2410] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:05:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:42.724366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbe293-9f00-4706-bbe7-4efd436a6b58 service glance] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:42.751856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbe293-9f00-4706-bbe7-4efd436a6b58 service glance] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:42.768173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565770eb-884c-4484-bba8-8a41769a5d68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:42.769425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565770eb-884c-4484-bba8-8a41769a5d68 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:42.769878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565770eb-884c-4484-bba8-8a41769a5d68 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:42.770313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565770eb-884c-4484-bba8-8a41769a5d68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:42.799719 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565770eb-884c-4484-bba8-8a41769a5d68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:42.807273 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565770eb-884c-4484-bba8-8a41769a5d68 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:42.817448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1213/2411] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:43.129970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab0ee8-7e51-4841-8199-20eb7d9fe126 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:43.130491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab0ee8-7e51-4841-8199-20eb7d9fe126 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:43.131261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab0ee8-7e51-4841-8199-20eb7d9fe126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:43.159077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e8ab0ee8-7e51-4841-8199-20eb7d9fe126 None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:43.160353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e8ab0ee8-7e51-4841-8199-20eb7d9fe126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:43.174784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1199/2412] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:05:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:43.199476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565770eb-884c-4484-bba8-8a41769a5d68 service swift] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:43.214061 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565770eb-884c-4484-bba8-8a41769a5d68 service swift] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:43.221619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9496993-53c7-48a5-9361-003e1af68a95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:43.222268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496993-53c7-48a5-9361-003e1af68a95 service swift] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:43.222829 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496993-53c7-48a5-9361-003e1af68a95 service swift] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:43.223092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496993-53c7-48a5-9361-003e1af68a95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:43.238730 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9496993-53c7-48a5-9361-003e1af68a95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:43.240401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9496993-53c7-48a5-9361-003e1af68a95 service swift] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:43.264369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1214/2413] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:49.151496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f72ae-5b16-4b79-9672-421404d63bd6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:49.152158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f72ae-5b16-4b79-9672-421404d63bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:49.152158 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f72ae-5b16-4b79-9672-421404d63bd6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:49.154013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1200/2414] 10.222.0.86 () {54 vars in 921 bytes} [Wed Aug 24 10:05:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:05:49.167117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f845cd4-0ca7-4f79-bf1a-43c4802c1df9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:49.167706 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f845cd4-0ca7-4f79-bf1a-43c4802c1df9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:49.167811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f845cd4-0ca7-4f79-bf1a-43c4802c1df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:49.200712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-9f845cd4-0ca7-4f79-bf1a-43c4802c1df9 None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:49.202377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9f845cd4-0ca7-4f79-bf1a-43c4802c1df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:49.406385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1215/2415] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Aug 24 10:05:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:53.151015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=85,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:55.510244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f72ae-5b16-4b79-9672-421404d63bd6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:55.512464 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-007f72ae-5b16-4b79-9672-421404d63bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:55.527864 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f72ae-5b16-4b79-9672-421404d63bd6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:55.531913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249066dd-5123-4970-9fea-539c1e06578f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '794dbf38682e4b4f939854eecbacaf07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:55.532668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249066dd-5123-4970-9fea-539c1e06578f service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:55.533048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249066dd-5123-4970-9fea-539c1e06578f service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:55.533440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249066dd-5123-4970-9fea-539c1e06578f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:55.549971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca34cdf-7084-4272-89f3-8fca3561ddd1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:55.550669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca34cdf-7084-4272-89f3-8fca3561ddd1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:55.550669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca34cdf-7084-4272-89f3-8fca3561ddd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:55.561446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249066dd-5123-4970-9fea-539c1e06578f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:55.562899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249066dd-5123-4970-9fea-539c1e06578f service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:55.575721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-0ca34cdf-7084-4272-89f3-8fca3561ddd1 None None] MFA Rules not processed for user `bea2aecac254429a823b8765feb6c767`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:55.577136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0ca34cdf-7084-4272-89f3-8fca3561ddd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:55.586410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1216/2417] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:05:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:05:55.588209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1201/2417] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:55.689630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a097089-f2ae-48f4-9f4b-d67757cb8ca9 None None] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:55.690818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a097089-f2ae-48f4-9f4b-d67757cb8ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:55.691123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a097089-f2ae-48f4-9f4b-d67757cb8ca9 None None] PATH_INFO: `/` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:55.692537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1217/2418] 10.222.0.86 () {54 vars in 921 bytes} [Wed Aug 24 10:05:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:05:55.698519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a59c-aa64-418a-915a-f6317ea3f54c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:55.699005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a59c-aa64-418a-915a-f6317ea3f54c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:55.699742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a59c-aa64-418a-915a-f6317ea3f54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:55.723591 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d097a59c-aa64-418a-915a-f6317ea3f54c None None] MFA Rules not processed for user `27cc091f3b094cb1a1e2650a3fbd7671`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:55.725659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d097a59c-aa64-418a-915a-f6317ea3f54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:55.734577 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1202/2419] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Aug 24 10:05:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:55.749566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a097089-f2ae-48f4-9f4b-d67757cb8ca9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:55.766503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a097089-f2ae-48f4-9f4b-d67757cb8ca9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:55.770639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4b1e0-0182-426b-9b30-7399305b189c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:55.771524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4b1e0-0182-426b-9b30-7399305b189c service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:55.771896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4b1e0-0182-426b-9b30-7399305b189c service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:55.772302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4b1e0-0182-426b-9b30-7399305b189c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:55.784204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4b1e0-0182-426b-9b30-7399305b189c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:55.785916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4b1e0-0182-426b-9b30-7399305b189c service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:55.796956 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1218/2420] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:55.945683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af0407-444f-4b7e-b455-371d29b3ff80 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:55.946194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af0407-444f-4b7e-b455-371d29b3ff80 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:55.946480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af0407-444f-4b7e-b455-371d29b3ff80 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:55.948500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1203/2421] 10.222.0.86 () {54 vars in 921 bytes} [Wed Aug 24 10:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:05:55.958608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911094f-d9ea-4906-8f64-d6419c4adc94 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:55.959144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911094f-d9ea-4906-8f64-d6419c4adc94 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:55.959547 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911094f-d9ea-4906-8f64-d6419c4adc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:55.986522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a911094f-d9ea-4906-8f64-d6419c4adc94 None None] MFA Rules not processed for user `98e11312493643b199fca04cb2d77853`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:55.988215 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a911094f-d9ea-4906-8f64-d6419c4adc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:56.000075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1219/2422] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Aug 24 10:05:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:56.011617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af0407-444f-4b7e-b455-371d29b3ff80 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:56.023348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af0407-444f-4b7e-b455-371d29b3ff80 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:56.027055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb9b69-8be9-42fe-b037-b59d7340df57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:56.027833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb9b69-8be9-42fe-b037-b59d7340df57 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:56.028214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb9b69-8be9-42fe-b037-b59d7340df57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:56.028565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb9b69-8be9-42fe-b037-b59d7340df57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:56.041121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb9b69-8be9-42fe-b037-b59d7340df57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:56.042683 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb9b69-8be9-42fe-b037-b59d7340df57 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:56.043904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580f787-6293-47e6-b7a6-e7d13018e3ec None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:56.044527 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580f787-6293-47e6-b7a6-e7d13018e3ec None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:56.044901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580f787-6293-47e6-b7a6-e7d13018e3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:56.059630 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1204/2423] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:56.070087 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6580f787-6293-47e6-b7a6-e7d13018e3ec None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:56.071759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6580f787-6293-47e6-b7a6-e7d13018e3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:56.079966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1220/2424] 10.222.0.86 () {58 vars in 967 bytes} [Wed Aug 24 10:05:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:56.095786 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb9b69-8be9-42fe-b037-b59d7340df57 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:56.108031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb9b69-8be9-42fe-b037-b59d7340df57 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:56.113620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283a2b0-4974-4b8f-82f6-d4e03c424408 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:56.114430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283a2b0-4974-4b8f-82f6-d4e03c424408 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:56.114872 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283a2b0-4974-4b8f-82f6-d4e03c424408 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:56.115338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283a2b0-4974-4b8f-82f6-d4e03c424408 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:56.126168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283a2b0-4974-4b8f-82f6-d4e03c424408 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:56.128395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283a2b0-4974-4b8f-82f6-d4e03c424408 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:56.137250 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1205/2425] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:56.516599 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccda16d-64ea-4ca4-b8ff-7b3d655cdc99 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:56.516954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccda16d-64ea-4ca4-b8ff-7b3d655cdc99 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:56.518252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccda16d-64ea-4ca4-b8ff-7b3d655cdc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:56.543003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-9ccda16d-64ea-4ca4-b8ff-7b3d655cdc99 None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:56.544776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9ccda16d-64ea-4ca4-b8ff-7b3d655cdc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:56.554933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1221/2426] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:05:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:56.577126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283a2b0-4974-4b8f-82f6-d4e03c424408 service swift] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:56.594140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283a2b0-4974-4b8f-82f6-d4e03c424408 service swift] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:56.613209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391ae79-db26-4ad4-99c3-0d7a5543170e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:56.616214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391ae79-db26-4ad4-99c3-0d7a5543170e service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:56.617496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391ae79-db26-4ad4-99c3-0d7a5543170e service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:56.617901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391ae79-db26-4ad4-99c3-0d7a5543170e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:56.632980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391ae79-db26-4ad4-99c3-0d7a5543170e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:56.634791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391ae79-db26-4ad4-99c3-0d7a5543170e service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:56.647716 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1206/2427] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 10:05:56.684008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46889ef-c697-4a94-bd9f-6cde70909e66 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:56.684415 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46889ef-c697-4a94-bd9f-6cde70909e66 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:56.684852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46889ef-c697-4a94-bd9f-6cde70909e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:56.705180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a46889ef-c697-4a94-bd9f-6cde70909e66 None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:56.706627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a46889ef-c697-4a94-bd9f-6cde70909e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:56.723758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1222/2428] 10.222.0.86 () {58 vars in 967 bytes} [Wed Aug 24 10:05:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:56.753798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391ae79-db26-4ad4-99c3-0d7a5543170e service swift] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:56.781564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391ae79-db26-4ad4-99c3-0d7a5543170e service swift] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:56.788087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00021982-e28e-4a56-8097-bb94844bc2a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:56.788981 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00021982-e28e-4a56-8097-bb94844bc2a3 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:56.789470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00021982-e28e-4a56-8097-bb94844bc2a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:56.789966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00021982-e28e-4a56-8097-bb94844bc2a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:56.799519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00021982-e28e-4a56-8097-bb94844bc2a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:56.801689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00021982-e28e-4a56-8097-bb94844bc2a3 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:56.811995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1207/2429] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:57.256794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43102486-c07c-465e-8218-345e8cbcce64 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:57.257792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43102486-c07c-465e-8218-345e8cbcce64 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:57.258154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43102486-c07c-465e-8218-345e8cbcce64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:57.292072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-43102486-c07c-465e-8218-345e8cbcce64 None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:57.293430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-43102486-c07c-465e-8218-345e8cbcce64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:57.301777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1223/2430] 10.222.0.86 () {58 vars in 967 bytes} [Wed Aug 24 10:05:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:57.315718 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00021982-e28e-4a56-8097-bb94844bc2a3 service swift] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:57.330108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00021982-e28e-4a56-8097-bb94844bc2a3 service swift] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:57.337562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88ff26-f970-4fcb-9196-fc6409418382 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:57.338176 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88ff26-f970-4fcb-9196-fc6409418382 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:57.338457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88ff26-f970-4fcb-9196-fc6409418382 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:57.338731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88ff26-f970-4fcb-9196-fc6409418382 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:57.350203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88ff26-f970-4fcb-9196-fc6409418382 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:57.357803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88ff26-f970-4fcb-9196-fc6409418382 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:57.372541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1208/2431] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:58.567664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c4a87-9ce9-4077-ab19-8371a9115994 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:58.567664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c4a87-9ce9-4077-ab19-8371a9115994 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:58.572520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c4a87-9ce9-4077-ab19-8371a9115994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:58.594971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-226c4a87-9ce9-4077-ab19-8371a9115994 None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:58.596251 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-226c4a87-9ce9-4077-ab19-8371a9115994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:58.604143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1224/2432] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:05:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:58.627106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88ff26-f970-4fcb-9196-fc6409418382 service swift] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:58.656345 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88ff26-f970-4fcb-9196-fc6409418382 service swift] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:58.664485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf4a69-b632-4b08-8822-93578d194551 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:58.665719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf4a69-b632-4b08-8822-93578d194551 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:58.665999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf4a69-b632-4b08-8822-93578d194551 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:58.666514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf4a69-b632-4b08-8822-93578d194551 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:58.681771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf4a69-b632-4b08-8822-93578d194551 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:58.684110 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf4a69-b632-4b08-8822-93578d194551 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:58.695946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1209/2433] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:59.589324 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c4a87-9ce9-4077-ab19-8371a9115994 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:59.592458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-226c4a87-9ce9-4077-ab19-8371a9115994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:59.628149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:59.712779 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf4a69-b632-4b08-8822-93578d194551 service swift] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:59.715303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-56bf4a69-b632-4b08-8822-93578d194551 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:59.788932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c4a87-9ce9-4077-ab19-8371a9115994 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:59.819059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9a717-6267-449a-9306-acaf5816e4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:59.820131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9a717-6267-449a-9306-acaf5816e4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:59.820678 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9a717-6267-449a-9306-acaf5816e4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:59.821123 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9a717-6267-449a-9306-acaf5816e4ae admin admin] PATH_INFO: `/v3/users/2063027ade874ab19c7df1655ebd6b85` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:59.823630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9a717-6267-449a-9306-acaf5816e4ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=2063027ade874ab19c7df1655ebd6b85)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:59.829292 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9a717-6267-449a-9306-acaf5816e4ae admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:59.923868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-11d9a717-6267-449a-9306-acaf5816e4ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2063027ade874ab19c7df1655ebd6b85'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:59.924694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-11d9a717-6267-449a-9306-acaf5816e4ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2063027ade874ab19c7df1655ebd6b85'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:59.925326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-11d9a717-6267-449a-9306-acaf5816e4ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2063027ade874ab19c7df1655ebd6b85'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:59.935919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf4a69-b632-4b08-8822-93578d194551 service swift] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:59.938244 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-11d9a717-6267-449a-9306-acaf5816e4ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2063027ade874ab19c7df1655ebd6b85'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:59.945340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:59.946218 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:59.947345 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:59.947789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] PATH_INFO: `/v3/users/731cf6df5bda42148e1420e365306445` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:59.950437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-11d9a717-6267-449a-9306-acaf5816e4ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2063027ade874ab19c7df1655ebd6b85'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:59.950694 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] RBAC: Authorizing `identity:delete_user(user_id=731cf6df5bda42148e1420e365306445)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:59.954327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:59.999089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1225/2434] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:05:59 2022] DELETE /identity/v3/users/2063027ade874ab19c7df1655ebd6b85 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:06:00.003796 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ee54a-a9ce-4e80-8822-dd5b119126d1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:00.004575 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ee54a-a9ce-4e80-8822-dd5b119126d1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:00.005095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ee54a-a9ce-4e80-8822-dd5b119126d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:00.053995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '731cf6df5bda42148e1420e365306445'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:00.057969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-947ee54a-a9ce-4e80-8822-dd5b119126d1 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:00.058807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '731cf6df5bda42148e1420e365306445'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:00.059724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-947ee54a-a9ce-4e80-8822-dd5b119126d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:00.060275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '731cf6df5bda42148e1420e365306445'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:00.089648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '731cf6df5bda42148e1420e365306445'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:00.115055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '731cf6df5bda42148e1420e365306445'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:00.116614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:00.165890 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1210/2435] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:05:59 2022] DELETE /identity/v3/users/731cf6df5bda42148e1420e365306445 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:06:00.228531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1226/2436] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:00.256136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:00.256136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:00.356710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8b00a-ba65-4745-93b0-d0e375d75026 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:00.357038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8b00a-ba65-4745-93b0-d0e375d75026 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:00.357319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8b00a-ba65-4745-93b0-d0e375d75026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:00.404726 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a6a8b00a-ba65-4745-93b0-d0e375d75026 None None] MFA Rules not processed for user `3ccfd2f7e35646b0a60b28b6807cee4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:00.406249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d81ce5-17cb-4cbf-b5d7-413b5446f16a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:00.406491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a6a8b00a-ba65-4745-93b0-d0e375d75026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:00.414706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca74c1a-f4b9-4a92-b05b-fa69a977dfd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:00.415787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca74c1a-f4b9-4a92-b05b-fa69a977dfd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:00.416247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca74c1a-f4b9-4a92-b05b-fa69a977dfd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:00.416672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca74c1a-f4b9-4a92-b05b-fa69a977dfd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:00.489997 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:00.542009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca74c1a-f4b9-4a92-b05b-fa69a977dfd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:00.545026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca74c1a-f4b9-4a92-b05b-fa69a977dfd4 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:00.561057 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1211/2437] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:06:00.575029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1227/2438] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:00.728685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca74c1a-f4b9-4a92-b05b-fa69a977dfd4 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:00.731595 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eca74c1a-f4b9-4a92-b05b-fa69a977dfd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:00.752985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca74c1a-f4b9-4a92-b05b-fa69a977dfd4 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:00.759289 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:00.760184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:00.760679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:00.761132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] PATH_INFO: `/v3/projects/f12553f79dbd4d75a291d3a07a8dbbba` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:00.762563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f12553f79dbd4d75a291d3a07a8dbbba)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:00.765325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:00.848196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f12553f79dbd4d75a291d3a07a8dbbba', 'request_id': 'req-0a443823-9977-4426-ad6d-f9bd99d63649'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:00.864531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f12553f79dbd4d75a291d3a07a8dbbba', 'request_id': 'req-0a443823-9977-4426-ad6d-f9bd99d63649'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:00.881450 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f12553f79dbd4d75a291d3a07a8dbbba', 'request_id': 'req-0a443823-9977-4426-ad6d-f9bd99d63649'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:00.901701 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] The token cache is being invalidate because project f12553f79dbd4d75a291d3a07a8dbbba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:06:00.901923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:00.902566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0a443823-9977-4426-ad6d-f9bd99d63649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:00.904576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1212/2439] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:06:00 2022] DELETE /identity/v3/projects/f12553f79dbd4d75a291d3a07a8dbbba => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:06:00.910450 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a096654-fe0d-4292-93a8-edc56117f252 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:00.911625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a096654-fe0d-4292-93a8-edc56117f252 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:00.913629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a096654-fe0d-4292-93a8-edc56117f252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:00.953148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-9a096654-fe0d-4292-93a8-edc56117f252 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:00.954665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9a096654-fe0d-4292-93a8-edc56117f252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:01.082908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1228/2440] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:01.091262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad4610-1027-490d-aa85-ff66eb251a41 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:01.092106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad4610-1027-490d-aa85-ff66eb251a41 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:01.092631 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad4610-1027-490d-aa85-ff66eb251a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:01.095461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a096654-fe0d-4292-93a8-edc56117f252 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:01.096957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9a096654-fe0d-4292-93a8-edc56117f252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:01.122768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a096654-fe0d-4292-93a8-edc56117f252 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:01.135037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e2ad4610-1027-490d-aa85-ff66eb251a41 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:01.136872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:01.137540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:01.137822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:01.138096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] PATH_INFO: `/v3/projects/b3d697163a8d4ee197f1b6b2856f08ae` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:01.139108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e2ad4610-1027-490d-aa85-ff66eb251a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:01.139639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3d697163a8d4ee197f1b6b2856f08ae)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:01.141660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:01.170144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1213/2441] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:01.178618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad4610-1027-490d-aa85-ff66eb251a41 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:01.197150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad4610-1027-490d-aa85-ff66eb251a41 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:01.214558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35a6ba-a090-4ff1-aa53-eddf1231ca0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:01.218885 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35a6ba-a090-4ff1-aa53-eddf1231ca0d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:01.219209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35a6ba-a090-4ff1-aa53-eddf1231ca0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:01.219489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35a6ba-a090-4ff1-aa53-eddf1231ca0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:01.221031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35a6ba-a090-4ff1-aa53-eddf1231ca0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:06:01.221339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35a6ba-a090-4ff1-aa53-eddf1231ca0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:01.225833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35a6ba-a090-4ff1-aa53-eddf1231ca0d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:01.228086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:01.237545 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3d697163a8d4ee197f1b6b2856f08ae', 'request_id': 'req-32ed8539-95df-4a3d-bf17-8da53189e256'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:01.260179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1214/2442] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:06:01.260905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3d697163a8d4ee197f1b6b2856f08ae', 'request_id': 'req-32ed8539-95df-4a3d-bf17-8da53189e256'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:01.266191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35a6ba-a090-4ff1-aa53-eddf1231ca0d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:01.281079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3d697163a8d4ee197f1b6b2856f08ae', 'request_id': 'req-32ed8539-95df-4a3d-bf17-8da53189e256'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:01.289034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35a6ba-a090-4ff1-aa53-eddf1231ca0d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:01.309071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] The token cache is being invalidate because project b3d697163a8d4ee197f1b6b2856f08ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:06:01.309450 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:01.312535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-32ed8539-95df-4a3d-bf17-8da53189e256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:01.319599 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1229/2443] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:06:01 2022] DELETE /identity/v3/projects/b3d697163a8d4ee197f1b6b2856f08ae => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:06:01.324742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572b9a1-5f86-46a1-9649-f6df743fa6b1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:01.325354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572b9a1-5f86-46a1-9649-f6df743fa6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:01.326100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572b9a1-5f86-46a1-9649-f6df743fa6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:01.401098 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5572b9a1-5f86-46a1-9649-f6df743fa6b1 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:01.402672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5572b9a1-5f86-46a1-9649-f6df743fa6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:01.526690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6f278-dafb-4939-8b9b-d15f4ec376bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:01.527362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6f278-dafb-4939-8b9b-d15f4ec376bb admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:01.527699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6f278-dafb-4939-8b9b-d15f4ec376bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:01.530404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6f278-dafb-4939-8b9b-d15f4ec376bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:01.530893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6f278-dafb-4939-8b9b-d15f4ec376bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:01.532898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6f278-dafb-4939-8b9b-d15f4ec376bb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:01.622767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1230/2444] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:06:01.624860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b311f-cdad-415b-8ccb-2bd9c362283a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:01.625233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b311f-cdad-415b-8ccb-2bd9c362283a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:01.625574 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b311f-cdad-415b-8ccb-2bd9c362283a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:01.636624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:01.637030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1215/2445] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:06:01 2022] POST /identity/v3/projects => generated 371 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:06:01.638419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8c10f-565c-44a9-a984-2b2082d58e98 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:01.638770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8c10f-565c-44a9-a984-2b2082d58e98 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:01.639095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8c10f-565c-44a9-a984-2b2082d58e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:01.660585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-429b311f-cdad-415b-8ccb-2bd9c362283a None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:01.663089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-429b311f-cdad-415b-8ccb-2bd9c362283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:01.682040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-44e8c10f-565c-44a9-a984-2b2082d58e98 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:01.693818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-44e8c10f-565c-44a9-a984-2b2082d58e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:01.793508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1231/2446] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Aug 24 10:06:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:01.796370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b311f-cdad-415b-8ccb-2bd9c362283a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:01.799490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-429b311f-cdad-415b-8ccb-2bd9c362283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:01.842178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1216/2447] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:01.844312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8c10f-565c-44a9-a984-2b2082d58e98 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:01.847901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-44e8c10f-565c-44a9-a984-2b2082d58e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:01.917379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b311f-cdad-415b-8ccb-2bd9c362283a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:01.922546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc9641-0c2f-4f04-9bbd-35bd54b53362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:01.923161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc9641-0c2f-4f04-9bbd-35bd54b53362 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:01.923528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc9641-0c2f-4f04-9bbd-35bd54b53362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:01.923759 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc9641-0c2f-4f04-9bbd-35bd54b53362 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:01.924433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc9641-0c2f-4f04-9bbd-35bd54b53362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:01.926399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc9641-0c2f-4f04-9bbd-35bd54b53362 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:01.955154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1232/2448] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:06:01 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:06:01.959192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc9641-0c2f-4f04-9bbd-35bd54b53362 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:01.973531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc9641-0c2f-4f04-9bbd-35bd54b53362 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:01.980280 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03d6a2-af58-4141-ad12-3c7c61609814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:01.981117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d6a2-af58-4141-ad12-3c7c61609814 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:01.981552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d6a2-af58-4141-ad12-3c7c61609814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:01.981974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d6a2-af58-4141-ad12-3c7c61609814 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:01.982981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03d6a2-af58-4141-ad12-3c7c61609814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:06:01.983719 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03d6a2-af58-4141-ad12-3c7c61609814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:01.986153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03d6a2-af58-4141-ad12-3c7c61609814 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:01.995299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8c10f-565c-44a9-a984-2b2082d58e98 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:02.004153 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7c2da-6e3c-4bc5-a396-0741657de623 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:02.004833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7c2da-6e3c-4bc5-a396-0741657de623 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.005187 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7c2da-6e3c-4bc5-a396-0741657de623 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.006615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1233/2449] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:06:02.012954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03d6a2-af58-4141-ad12-3c7c61609814 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:02.014684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7c2da-6e3c-4bc5-a396-0741657de623 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:02.026181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7c2da-6e3c-4bc5-a396-0741657de623 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:02.027188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03d6a2-af58-4141-ad12-3c7c61609814 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:02.027817 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7c2da-6e3c-4bc5-a396-0741657de623 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:02.032999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cd3b4-8253-47d3-9d04-71c506a2ff01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:02.033905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cd3b4-8253-47d3-9d04-71c506a2ff01 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.034345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cd3b4-8253-47d3-9d04-71c506a2ff01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.034770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cd3b4-8253-47d3-9d04-71c506a2ff01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:02.035581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cd3b4-8253-47d3-9d04-71c506a2ff01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:02.038101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cd3b4-8253-47d3-9d04-71c506a2ff01 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:02.046876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1217/2450] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:06:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:06:02.048117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7c2da-6e3c-4bc5-a396-0741657de623 service swift] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:02.060878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1234/2451] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:06:02.063124 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cd3b4-8253-47d3-9d04-71c506a2ff01 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:02.075242 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7c2da-6e3c-4bc5-a396-0741657de623 service swift] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:02.084989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a3296-478c-47f2-8544-eea2fea824fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:02.085244 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cd3b4-8253-47d3-9d04-71c506a2ff01 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:02.087169 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a3296-478c-47f2-8544-eea2fea824fe admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.089023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a3296-478c-47f2-8544-eea2fea824fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.089602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a3296-478c-47f2-8544-eea2fea824fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:02.089917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e06db4-9b0c-4657-87c2-66df490bcd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:02.090726 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e06db4-9b0c-4657-87c2-66df490bcd5f admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.091014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e06db4-9b0c-4657-87c2-66df490bcd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.091358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e06db4-9b0c-4657-87c2-66df490bcd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:02.091793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a3296-478c-47f2-8544-eea2fea824fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:02.092061 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e06db4-9b0c-4657-87c2-66df490bcd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:02.094626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e06db4-9b0c-4657-87c2-66df490bcd5f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:02.096371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a3296-478c-47f2-8544-eea2fea824fe admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:02.133796 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:02.142744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1235/2452] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:06:02.144228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e06db4-9b0c-4657-87c2-66df490bcd5f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:02.155906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e06db4-9b0c-4657-87c2-66df490bcd5f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:02.165161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e636625-b5fd-4345-99ab-55ec747c7122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:02.165859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e636625-b5fd-4345-99ab-55ec747c7122 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.166148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e636625-b5fd-4345-99ab-55ec747c7122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.166629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e636625-b5fd-4345-99ab-55ec747c7122 admin admin] PATH_INFO: `/v3/projects/98dbf124c1af4b7983c8f702b2c5fa52/users/9720b76a2c924488a2664a7179fb2ead/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:02.187942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1218/2453] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:06:02 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:06:02.193530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a3296-478c-47f2-8544-eea2fea824fe admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:02.245392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e636625-b5fd-4345-99ab-55ec747c7122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98dbf124c1af4b7983c8f702b2c5fa52, user_id=9720b76a2c924488a2664a7179fb2ead, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:02.252253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e636625-b5fd-4345-99ab-55ec747c7122 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:02.282144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1236/2454] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:06:02 2022] PUT /identity/v3/projects/98dbf124c1af4b7983c8f702b2c5fa52/users/9720b76a2c924488a2664a7179fb2ead/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:06:02.296527 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86f94e-7bd5-4c8e-bf9e-431a70f562ec None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.296819 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86f94e-7bd5-4c8e-bf9e-431a70f562ec None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.297099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86f94e-7bd5-4c8e-bf9e-431a70f562ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:02.373410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-aa86f94e-7bd5-4c8e-bf9e-431a70f562ec None None] MFA Rules not processed for user `9720b76a2c924488a2664a7179fb2ead`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:02.374866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-aa86f94e-7bd5-4c8e-bf9e-431a70f562ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:02.381814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a3296-478c-47f2-8544-eea2fea824fe admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:02.441209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:02.503159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1237/2455] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:02.586666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ee2a2-a796-43bc-9c5a-4a969fd9b4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:02.590077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee2a2-a796-43bc-9c5a-4a969fd9b4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.590534 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee2a2-a796-43bc-9c5a-4a969fd9b4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.591536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee2a2-a796-43bc-9c5a-4a969fd9b4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:02.592456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ee2a2-a796-43bc-9c5a-4a969fd9b4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:02.603447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ee2a2-a796-43bc-9c5a-4a969fd9b4a2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:02.684419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1219/2456] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:06:02 2022] POST /identity/v3/users => generated 292 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:06:02.690439 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86f94e-7bd5-4c8e-bf9e-431a70f562ec None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:02.713067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86f94e-7bd5-4c8e-bf9e-431a70f562ec None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:02.719196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908e9f2-b2d1-41a3-a1e3-48c5f6f29881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:02.720083 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908e9f2-b2d1-41a3-a1e3-48c5f6f29881 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.721495 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908e9f2-b2d1-41a3-a1e3-48c5f6f29881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.721925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908e9f2-b2d1-41a3-a1e3-48c5f6f29881 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:02.723874 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908e9f2-b2d1-41a3-a1e3-48c5f6f29881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:02.728788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908e9f2-b2d1-41a3-a1e3-48c5f6f29881 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:02.744792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1238/2457] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:06:02.749140 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ee2a2-a796-43bc-9c5a-4a969fd9b4a2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:02.762239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ee2a2-a796-43bc-9c5a-4a969fd9b4a2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:02.769807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dc06a-4e48-47e2-984e-7110656026bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:02.770637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dc06a-4e48-47e2-984e-7110656026bf admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.771099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dc06a-4e48-47e2-984e-7110656026bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.771448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dc06a-4e48-47e2-984e-7110656026bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:02.772437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dc06a-4e48-47e2-984e-7110656026bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:02.774136 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dc06a-4e48-47e2-984e-7110656026bf admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:02.794934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1220/2458] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:06:02.799616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908e9f2-b2d1-41a3-a1e3-48c5f6f29881 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:02.815726 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908e9f2-b2d1-41a3-a1e3-48c5f6f29881 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:02.834947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f693b6-2a15-4c56-8863-c805d2bd0662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:02.836458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f693b6-2a15-4c56-8863-c805d2bd0662 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.836837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f693b6-2a15-4c56-8863-c805d2bd0662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.838108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f693b6-2a15-4c56-8863-c805d2bd0662 admin admin] PATH_INFO: `/v3/projects/f1eac6259f65441d9abbabe2a04afad1/users/4ce9eb7417a7482cb9cdaed16cb6aeb9/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:02.889707 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f693b6-2a15-4c56-8863-c805d2bd0662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1eac6259f65441d9abbabe2a04afad1, user_id=4ce9eb7417a7482cb9cdaed16cb6aeb9, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:02.909345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f693b6-2a15-4c56-8863-c805d2bd0662 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:02.972474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1239/2459] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:06:02 2022] PUT /identity/v3/projects/f1eac6259f65441d9abbabe2a04afad1/users/4ce9eb7417a7482cb9cdaed16cb6aeb9/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:06:02.978907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494787c4-c755-43aa-b6b0-e6eecf7c9fb4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.979698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494787c4-c755-43aa-b6b0-e6eecf7c9fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.979798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494787c4-c755-43aa-b6b0-e6eecf7c9fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:03.009115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-494787c4-c755-43aa-b6b0-e6eecf7c9fb4 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:03.011165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-494787c4-c755-43aa-b6b0-e6eecf7c9fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:03.107291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1221/2460] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:03.119038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f693b6-2a15-4c56-8863-c805d2bd0662 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:03.121650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-88f693b6-2a15-4c56-8863-c805d2bd0662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:03.251395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f693b6-2a15-4c56-8863-c805d2bd0662 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:03.256227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2977f88-4b0f-403f-a8af-6ef3ba16fe81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:03.257121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2977f88-4b0f-403f-a8af-6ef3ba16fe81 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:03.257487 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2977f88-4b0f-403f-a8af-6ef3ba16fe81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:03.257800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2977f88-4b0f-403f-a8af-6ef3ba16fe81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:03.275530 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2977f88-4b0f-403f-a8af-6ef3ba16fe81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:03.277378 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2977f88-4b0f-403f-a8af-6ef3ba16fe81 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:03.296319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1240/2461] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:06:04.279142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96292612-483d-464d-aae2-8965120b7d89 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:04.279754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96292612-483d-464d-aae2-8965120b7d89 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:04.280402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96292612-483d-464d-aae2-8965120b7d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:04.332124 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:04.336484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-96292612-483d-464d-aae2-8965120b7d89 None None] MFA Rules not processed for user `4ce9eb7417a7482cb9cdaed16cb6aeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:04.342035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-96292612-483d-464d-aae2-8965120b7d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:04.537456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1222/2462] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:04.550167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2977f88-4b0f-403f-a8af-6ef3ba16fe81 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:04.552871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e2977f88-4b0f-403f-a8af-6ef3ba16fe81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:04.674654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2977f88-4b0f-403f-a8af-6ef3ba16fe81 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:04.678843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845e32-e9ac-4d8b-b03f-646c71c5bfed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:04.679451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845e32-e9ac-4d8b-b03f-646c71c5bfed service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:04.679729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845e32-e9ac-4d8b-b03f-646c71c5bfed service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:04.680001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845e32-e9ac-4d8b-b03f-646c71c5bfed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:04.705343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86845e32-e9ac-4d8b-b03f-646c71c5bfed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:04.707326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86845e32-e9ac-4d8b-b03f-646c71c5bfed service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:04.719653 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1241/2463] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:06:06.356428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd717ab7-37b6-4b57-b645-2a95a2996694 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:06.357469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd717ab7-37b6-4b57-b645-2a95a2996694 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:06.357772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd717ab7-37b6-4b57-b645-2a95a2996694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:06.380300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cd717ab7-37b6-4b57-b645-2a95a2996694 None None] MFA Rules not processed for user `4ce9eb7417a7482cb9cdaed16cb6aeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:06.381695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cd717ab7-37b6-4b57-b645-2a95a2996694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:06.389292 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1223/2464] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:12.514480 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845e32-e9ac-4d8b-b03f-646c71c5bfed service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:12.515952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-86845e32-e9ac-4d8b-b03f-646c71c5bfed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:12.559409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:12.574942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845e32-e9ac-4d8b-b03f-646c71c5bfed service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:12.579339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f2e2d-f8d5-4ef5-8036-16ff9b4c4b1f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '794dbf38682e4b4f939854eecbacaf07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:12.579944 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2e2d-f8d5-4ef5-8036-16ff9b4c4b1f service glance] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:12.580270 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2e2d-f8d5-4ef5-8036-16ff9b4c4b1f service glance] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:12.580543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2e2d-f8d5-4ef5-8036-16ff9b4c4b1f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:12.648758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f2e2d-f8d5-4ef5-8036-16ff9b4c4b1f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:12.650221 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f2e2d-f8d5-4ef5-8036-16ff9b4c4b1f service glance] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:12.659850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1242/2465] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:06:16.374339 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=29 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:18.752234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd717ab7-37b6-4b57-b645-2a95a2996694 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:18.753529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cd717ab7-37b6-4b57-b645-2a95a2996694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:18.854503 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd717ab7-37b6-4b57-b645-2a95a2996694 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:18.859819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf215d9-9a38-4cd2-a295-aa27282538f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:18.860462 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf215d9-9a38-4cd2-a295-aa27282538f3 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:18.860745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf215d9-9a38-4cd2-a295-aa27282538f3 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:18.861020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf215d9-9a38-4cd2-a295-aa27282538f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:18.862763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-baf215d9-9a38-4cd2-a295-aa27282538f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:18.876127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf215d9-9a38-4cd2-a295-aa27282538f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:18.877831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf215d9-9a38-4cd2-a295-aa27282538f3 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:18.888760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1224/2466] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:06:22.655142 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=30 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:28.884410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=32 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:42.930827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842736b-1881-4bae-bfef-4f54a6815627 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:42.931378 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842736b-1881-4bae-bfef-4f54a6815627 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:42.931478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842736b-1881-4bae-bfef-4f54a6815627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:42.955957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1842736b-1881-4bae-bfef-4f54a6815627 None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:42.957276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1842736b-1881-4bae-bfef-4f54a6815627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:43.033923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1243/2467] 10.222.0.86 () {58 vars in 967 bytes} [Wed Aug 24 10:06:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:43.056182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf215d9-9a38-4cd2-a295-aa27282538f3 service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:43.121413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf215d9-9a38-4cd2-a295-aa27282538f3 service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:43.126357 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2ac87-1921-4d17-96f9-f71608fb8582 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:43.126949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2ac87-1921-4d17-96f9-f71608fb8582 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:43.127228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2ac87-1921-4d17-96f9-f71608fb8582 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:43.127508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2ac87-1921-4d17-96f9-f71608fb8582 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:43.135298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2ac87-1921-4d17-96f9-f71608fb8582 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:43.136993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2ac87-1921-4d17-96f9-f71608fb8582 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:43.144821 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1225/2468] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:06:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:06:46.010781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842736b-1881-4bae-bfef-4f54a6815627 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:46.021963 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842736b-1881-4bae-bfef-4f54a6815627 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:46.026514 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:46.027129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:46.027413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:46.027691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] PATH_INFO: `/v3/users/9720b76a2c924488a2664a7179fb2ead` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:46.028865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9720b76a2c924488a2664a7179fb2ead)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:46.030807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:46.079203 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9720b76a2c924488a2664a7179fb2ead'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:46.079863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9720b76a2c924488a2664a7179fb2ead'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:46.080507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9720b76a2c924488a2664a7179fb2ead'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:46.089059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9720b76a2c924488a2664a7179fb2ead'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:46.096477 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9720b76a2c924488a2664a7179fb2ead'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:46.115470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1244/2469] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:06:46 2022] DELETE /identity/v3/users/9720b76a2c924488a2664a7179fb2ead => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:06:46.119676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5577296-dc6e-448c-86af-226102aac36b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:46.120204 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5577296-dc6e-448c-86af-226102aac36b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:46.120310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5577296-dc6e-448c-86af-226102aac36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:46.138374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e5577296-dc6e-448c-86af-226102aac36b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:46.139572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e5577296-dc6e-448c-86af-226102aac36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:46.204180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1226/2470] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:46.213905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:46.215542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:46.276231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898be1a-da7d-46c3-8807-8e40c109609f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:46.279847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a339add-2875-44a4-adce-ffc79f41de54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:46.280482 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a339add-2875-44a4-adce-ffc79f41de54 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:46.280778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a339add-2875-44a4-adce-ffc79f41de54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:46.281031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a339add-2875-44a4-adce-ffc79f41de54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:46.290361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a339add-2875-44a4-adce-ffc79f41de54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:46.291927 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a339add-2875-44a4-adce-ffc79f41de54 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:46.302640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1245/2471] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:06:46.573559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5577296-dc6e-448c-86af-226102aac36b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:46.575001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e5577296-dc6e-448c-86af-226102aac36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:46.587846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5577296-dc6e-448c-86af-226102aac36b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:46.592956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:46.593590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:46.593903 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:46.594180 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] PATH_INFO: `/v3/projects/98dbf124c1af4b7983c8f702b2c5fa52` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:46.595369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=98dbf124c1af4b7983c8f702b2c5fa52)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:46.598065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:46.649151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98dbf124c1af4b7983c8f702b2c5fa52', 'request_id': 'req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:46.656413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98dbf124c1af4b7983c8f702b2c5fa52', 'request_id': 'req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:46.663496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98dbf124c1af4b7983c8f702b2c5fa52', 'request_id': 'req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:46.677807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] The token cache is being invalidate because project 98dbf124c1af4b7983c8f702b2c5fa52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:06:46.678350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:46.678987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a0a9d782-01ec-4691-a5ac-936bc53cbd6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:46.683969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1227/2472] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:06:46 2022] DELETE /identity/v3/projects/98dbf124c1af4b7983c8f702b2c5fa52 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:06:46.685559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720a295-fe9c-4b52-b4a5-a445a3a4558e None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:46.685865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720a295-fe9c-4b52-b4a5-a445a3a4558e None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:46.686126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720a295-fe9c-4b52-b4a5-a445a3a4558e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:46.706807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4720a295-fe9c-4b52-b4a5-a445a3a4558e None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:46.707974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4720a295-fe9c-4b52-b4a5-a445a3a4558e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:46.723619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:46.790395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1246/2473] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:46.797485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0940bcb-976a-42d8-95e3-7e14704d2c1a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:46.798048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0940bcb-976a-42d8-95e3-7e14704d2c1a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:46.798323 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0940bcb-976a-42d8-95e3-7e14704d2c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:46.820409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a0940bcb-976a-42d8-95e3-7e14704d2c1a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:46.821621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a0940bcb-976a-42d8-95e3-7e14704d2c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:46.829743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1228/2474] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:06:46.834625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720a295-fe9c-4b52-b4a5-a445a3a4558e None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:46.844291 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720a295-fe9c-4b52-b4a5-a445a3a4558e None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:46.849686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901d65d-344d-414b-9256-9566e0ef4a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:46.850284 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901d65d-344d-414b-9256-9566e0ef4a53 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:46.850568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901d65d-344d-414b-9256-9566e0ef4a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:46.850843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901d65d-344d-414b-9256-9566e0ef4a53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:46.851487 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901d65d-344d-414b-9256-9566e0ef4a53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:06:46.851790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901d65d-344d-414b-9256-9566e0ef4a53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:46.853711 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901d65d-344d-414b-9256-9566e0ef4a53 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:46.866811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1247/2475] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:06:46.870907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0940bcb-976a-42d8-95e3-7e14704d2c1a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:46.880262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0940bcb-976a-42d8-95e3-7e14704d2c1a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:46.885194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac205fc0-bf34-4be3-836d-9a26aac0cca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:46.885873 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205fc0-bf34-4be3-836d-9a26aac0cca6 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:46.886143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205fc0-bf34-4be3-836d-9a26aac0cca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:46.886414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205fc0-bf34-4be3-836d-9a26aac0cca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:46.887068 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac205fc0-bf34-4be3-836d-9a26aac0cca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:46.889017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac205fc0-bf34-4be3-836d-9a26aac0cca6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:46.893719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:46.917971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1229/2476] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:06:46 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:06:46.922432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901d65d-344d-414b-9256-9566e0ef4a53 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:46.933228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901d65d-344d-414b-9256-9566e0ef4a53 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:46.991332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2707d4-73a4-4aeb-aebf-3f961c7f63e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:46.991939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2707d4-73a4-4aeb-aebf-3f961c7f63e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:46.992246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2707d4-73a4-4aeb-aebf-3f961c7f63e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:46.992519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2707d4-73a4-4aeb-aebf-3f961c7f63e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:46.993165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2707d4-73a4-4aeb-aebf-3f961c7f63e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:46.995097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2707d4-73a4-4aeb-aebf-3f961c7f63e0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:47.019856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1248/2477] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:06:46 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:06:47.020974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac205fc0-bf34-4be3-836d-9a26aac0cca6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:47.031293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac205fc0-bf34-4be3-836d-9a26aac0cca6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:47.036511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c7ed2-fd18-4ac6-9070-90d7f0659fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:47.037111 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c7ed2-fd18-4ac6-9070-90d7f0659fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:47.037391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c7ed2-fd18-4ac6-9070-90d7f0659fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:47.037693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c7ed2-fd18-4ac6-9070-90d7f0659fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:47.038337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c7ed2-fd18-4ac6-9070-90d7f0659fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:47.039933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c7ed2-fd18-4ac6-9070-90d7f0659fa9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:47.051002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1230/2478] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:06:47.055073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2707d4-73a4-4aeb-aebf-3f961c7f63e0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:47.066213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2707d4-73a4-4aeb-aebf-3f961c7f63e0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:47.071105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a5689-a4d7-4f19-8450-4cc46dd8aa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:47.071869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a5689-a4d7-4f19-8450-4cc46dd8aa08 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:47.072172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a5689-a4d7-4f19-8450-4cc46dd8aa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:47.072452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a5689-a4d7-4f19-8450-4cc46dd8aa08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:47.073100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a5689-a4d7-4f19-8450-4cc46dd8aa08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:47.074737 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a5689-a4d7-4f19-8450-4cc46dd8aa08 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:47.085880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1249/2479] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:06:47.090146 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c7ed2-fd18-4ac6-9070-90d7f0659fa9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:47.100808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c7ed2-fd18-4ac6-9070-90d7f0659fa9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:47.105742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a2b70-d3fe-4dbf-9941-e872d9af16f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:47.106397 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a2b70-d3fe-4dbf-9941-e872d9af16f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:47.106666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a2b70-d3fe-4dbf-9941-e872d9af16f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:47.106943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a2b70-d3fe-4dbf-9941-e872d9af16f7 admin admin] PATH_INFO: `/v3/projects/965a2fd0658145e18b76e83fa373f54d/users/b41fdcf2b8e34dfaa997a0c1fb733466/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:47.122706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a2b70-d3fe-4dbf-9941-e872d9af16f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=965a2fd0658145e18b76e83fa373f54d, user_id=b41fdcf2b8e34dfaa997a0c1fb733466, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:47.125400 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a2b70-d3fe-4dbf-9941-e872d9af16f7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:47.143454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1231/2480] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:06:47 2022] PUT /identity/v3/projects/965a2fd0658145e18b76e83fa373f54d/users/b41fdcf2b8e34dfaa997a0c1fb733466/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:06:47.153459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a4bba-15de-4ad8-b034-46b5dba16f2c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:47.153738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a4bba-15de-4ad8-b034-46b5dba16f2c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:47.154018 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a4bba-15de-4ad8-b034-46b5dba16f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:47.183543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e30a4bba-15de-4ad8-b034-46b5dba16f2c None None] MFA Rules not processed for user `b41fdcf2b8e34dfaa997a0c1fb733466`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:47.184778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e30a4bba-15de-4ad8-b034-46b5dba16f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:47.246929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1250/2481] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:06:47.258039 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a2b70-d3fe-4dbf-9941-e872d9af16f7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:47.259938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e00a2b70-d3fe-4dbf-9941-e872d9af16f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:47.316822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a2b70-d3fe-4dbf-9941-e872d9af16f7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:47.320452 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9b36e-f256-4404-a0bc-adca58fdd99a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:47.321049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9b36e-f256-4404-a0bc-adca58fdd99a service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:47.321326 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9b36e-f256-4404-a0bc-adca58fdd99a service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:47.321596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9b36e-f256-4404-a0bc-adca58fdd99a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:47.329661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9b36e-f256-4404-a0bc-adca58fdd99a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:47.331607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9b36e-f256-4404-a0bc-adca58fdd99a service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:47.340631 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1232/2482] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:06:47.370082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a4bba-15de-4ad8-b034-46b5dba16f2c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:47.387428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:47.433828 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a4bba-15de-4ad8-b034-46b5dba16f2c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:47.438453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:47.439073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:47.439356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:47.439625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] PATH_INFO: `/v3/users/b41fdcf2b8e34dfaa997a0c1fb733466` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:47.440783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41fdcf2b8e34dfaa997a0c1fb733466)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:47.442734 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:47.490027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41fdcf2b8e34dfaa997a0c1fb733466'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:47.490699 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41fdcf2b8e34dfaa997a0c1fb733466'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:47.491329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41fdcf2b8e34dfaa997a0c1fb733466'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:47.498629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41fdcf2b8e34dfaa997a0c1fb733466'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:47.505818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41fdcf2b8e34dfaa997a0c1fb733466'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:47.523242 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1251/2483] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:06:47 2022] DELETE /identity/v3/users/b41fdcf2b8e34dfaa997a0c1fb733466 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:06:47.527801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bba30c-674b-4e86-8c9f-670effd347df None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:47.528125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bba30c-674b-4e86-8c9f-670effd347df None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:47.528405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bba30c-674b-4e86-8c9f-670effd347df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:47.547511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-38bba30c-674b-4e86-8c9f-670effd347df None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:06:47.548802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-38bba30c-674b-4e86-8c9f-670effd347df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:47.610678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1233/2484] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:06:47.620228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:47.621703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:47.682213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef748de2-d64e-4066-852c-2397249c675f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:47.686207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419757ca-ef11-46d9-9c8e-581583e1eb8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:47.686825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419757ca-ef11-46d9-9c8e-581583e1eb8b service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:47.687103 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419757ca-ef11-46d9-9c8e-581583e1eb8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:47.687373 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419757ca-ef11-46d9-9c8e-581583e1eb8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:47.697552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419757ca-ef11-46d9-9c8e-581583e1eb8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:47.699026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419757ca-ef11-46d9-9c8e-581583e1eb8b service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:47.708990 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1252/2485] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:06:47.978168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bba30c-674b-4e86-8c9f-670effd347df None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:47.979665 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-38bba30c-674b-4e86-8c9f-670effd347df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:47.994764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bba30c-674b-4e86-8c9f-670effd347df None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:47.999614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:48.000389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:48.000742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:48.001086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] PATH_INFO: `/v3/projects/965a2fd0658145e18b76e83fa373f54d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:48.002941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] RBAC: Authorizing `identity:delete_project(project_id=965a2fd0658145e18b76e83fa373f54d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:48.006123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:48.052334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:48.054019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '965a2fd0658145e18b76e83fa373f54d', 'request_id': 'req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:48.061566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '965a2fd0658145e18b76e83fa373f54d', 'request_id': 'req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:48.069352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '965a2fd0658145e18b76e83fa373f54d', 'request_id': 'req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:48.083137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] The token cache is being invalidate because project 965a2fd0658145e18b76e83fa373f54d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:06:48.083425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:48.084063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:48.085560 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1234/2486] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:06:47 2022] DELETE /identity/v3/projects/965a2fd0658145e18b76e83fa373f54d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:06:50.856309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419757ca-ef11-46d9-9c8e-581583e1eb8b service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:50.857837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-419757ca-ef11-46d9-9c8e-581583e1eb8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:50.869153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419757ca-ef11-46d9-9c8e-581583e1eb8b service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:50.902913 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:50.920917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c99db-0769-4661-8b62-7cbd772748f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:50.921515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c99db-0769-4661-8b62-7cbd772748f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:50.921794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c99db-0769-4661-8b62-7cbd772748f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:50.922064 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c99db-0769-4661-8b62-7cbd772748f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:50.923801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-609c99db-0769-4661-8b62-7cbd772748f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:50.987469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c99db-0769-4661-8b62-7cbd772748f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:50.988953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c99db-0769-4661-8b62-7cbd772748f5 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:51.008610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1253/2487] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:06:53.444855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:53.446485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:53.461229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e00cb-240d-4d8f-8f15-5ae7bfe7b01f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:53.464856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb646c0e-3e5e-47cd-9523-f81a405c6ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:53.465453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb646c0e-3e5e-47cd-9523-f81a405c6ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:53.465739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb646c0e-3e5e-47cd-9523-f81a405c6ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:53.466009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb646c0e-3e5e-47cd-9523-f81a405c6ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:53.467743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cb646c0e-3e5e-47cd-9523-f81a405c6ea3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:53.536616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb646c0e-3e5e-47cd-9523-f81a405c6ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:53.538054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb646c0e-3e5e-47cd-9523-f81a405c6ea3 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:53.547644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1235/2488] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:07:01.003684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=33 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:03.543440 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=37,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:29.707045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61515072-f44b-454e-bcd9-e84db8f0dc11 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:29.707598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61515072-f44b-454e-bcd9-e84db8f0dc11 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:29.707598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61515072-f44b-454e-bcd9-e84db8f0dc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:29.726569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-61515072-f44b-454e-bcd9-e84db8f0dc11 None None] MFA Rules not processed for user `46c01f8b99e24be5b6c2d73b9b1bd9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:07:29.727757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-61515072-f44b-454e-bcd9-e84db8f0dc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:29.781875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1254/2489] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:07:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:07:39.775199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=25 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:58.800726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb646c0e-3e5e-47cd-9523-f81a405c6ea3 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:07:58.803009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cb646c0e-3e5e-47cd-9523-f81a405c6ea3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:58.815603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb646c0e-3e5e-47cd-9523-f81a405c6ea3 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:07:58.819518 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6518fc-34b6-47cc-b0cc-64d12ffa4508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:07:58.820155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6518fc-34b6-47cc-b0cc-64d12ffa4508 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:58.820436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6518fc-34b6-47cc-b0cc-64d12ffa4508 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:58.820708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6518fc-34b6-47cc-b0cc-64d12ffa4508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:58.822622 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1b6518fc-34b6-47cc-b0cc-64d12ffa4508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:58.903271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6518fc-34b6-47cc-b0cc-64d12ffa4508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:07:58.904688 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6518fc-34b6-47cc-b0cc-64d12ffa4508 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:07:58.916566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1236/2490] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:07:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:06.832908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61515072-f44b-454e-bcd9-e84db8f0dc11 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:06.834342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-61515072-f44b-454e-bcd9-e84db8f0dc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:06.855380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61515072-f44b-454e-bcd9-e84db8f0dc11 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:06.860952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefefbd9-2574-4817-818f-0502bcc449f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:06.861552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefefbd9-2574-4817-818f-0502bcc449f9 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:06.861839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefefbd9-2574-4817-818f-0502bcc449f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:06.862114 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefefbd9-2574-4817-818f-0502bcc449f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:06.863833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-eefefbd9-2574-4817-818f-0502bcc449f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:06.922138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefefbd9-2574-4817-818f-0502bcc449f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:06.924227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefefbd9-2574-4817-818f-0502bcc449f9 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:06.934467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1255/2491] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:08.910964 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=34 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:09.472866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6518fc-34b6-47cc-b0cc-64d12ffa4508 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:09.481829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6518fc-34b6-47cc-b0cc-64d12ffa4508 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:09.485486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fdc55-6bc2-4530-b1d9-0a390273924c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:09.486075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fdc55-6bc2-4530-b1d9-0a390273924c service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:09.486356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fdc55-6bc2-4530-b1d9-0a390273924c service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:09.486631 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fdc55-6bc2-4530-b1d9-0a390273924c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:09.488392 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-df0fdc55-6bc2-4530-b1d9-0a390273924c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:09.501916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fdc55-6bc2-4530-b1d9-0a390273924c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:09.503345 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fdc55-6bc2-4530-b1d9-0a390273924c service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:09.514732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1237/2492] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:16.929806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=28 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:19.509896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:20.791764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefefbd9-2574-4817-818f-0502bcc449f9 service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:20.802553 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefefbd9-2574-4817-818f-0502bcc449f9 service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:20.806152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffa0db-f7f9-4978-a3c6-2941b4994743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:20.806765 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffa0db-f7f9-4978-a3c6-2941b4994743 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:20.807043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffa0db-f7f9-4978-a3c6-2941b4994743 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:20.807320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffa0db-f7f9-4978-a3c6-2941b4994743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:20.809155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b3ffa0db-f7f9-4978-a3c6-2941b4994743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:20.819589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffa0db-f7f9-4978-a3c6-2941b4994743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:20.821614 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffa0db-f7f9-4978-a3c6-2941b4994743 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:20.829884 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1256/2493] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:30.826225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:04.057650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fdc55-6bc2-4530-b1d9-0a390273924c service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:04.070411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fdc55-6bc2-4530-b1d9-0a390273924c service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:04.077001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c3786-9f4b-46ed-8241-342df750b9d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:04.077736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c3786-9f4b-46ed-8241-342df750b9d5 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:04.078095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c3786-9f4b-46ed-8241-342df750b9d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:04.078494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c3786-9f4b-46ed-8241-342df750b9d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:04.080409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3b5c3786-9f4b-46ed-8241-342df750b9d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:04.093988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c3786-9f4b-46ed-8241-342df750b9d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:04.095521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c3786-9f4b-46ed-8241-342df750b9d5 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:04.107343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1238/2494] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:14.103121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:03.149423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffa0db-f7f9-4978-a3c6-2941b4994743 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:03.149423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b3ffa0db-f7f9-4978-a3c6-2941b4994743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:03.205062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffa0db-f7f9-4978-a3c6-2941b4994743 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:03.208708 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651a8d5-486b-42bc-9c93-a18d2186539d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:03.209307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651a8d5-486b-42bc-9c93-a18d2186539d service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:03.209589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651a8d5-486b-42bc-9c93-a18d2186539d service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:03.209863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651a8d5-486b-42bc-9c93-a18d2186539d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:03.218657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0651a8d5-486b-42bc-9c93-a18d2186539d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:03.241403 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c3786-9f4b-46ed-8241-342df750b9d5 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:03.242830 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3b5c3786-9f4b-46ed-8241-342df750b9d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:03.254239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c3786-9f4b-46ed-8241-342df750b9d5 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:03.257862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cb0bd-0b8b-4215-9924-10535efca0b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:03.258463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb0bd-0b8b-4215-9924-10535efca0b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:03.258746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb0bd-0b8b-4215-9924-10535efca0b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:03.259021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb0bd-0b8b-4215-9924-10535efca0b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:03.260934 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d87cb0bd-0b8b-4215-9924-10535efca0b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:03.272470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cb0bd-0b8b-4215-9924-10535efca0b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:03.274089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cb0bd-0b8b-4215-9924-10535efca0b6 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:03.275776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651a8d5-486b-42bc-9c93-a18d2186539d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:03.277252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651a8d5-486b-42bc-9c93-a18d2186539d service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:03.284007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1239/2495] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:03.286152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1257/2496] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:13.279736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:13.282001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=48 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:03.411880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cb0bd-0b8b-4215-9924-10535efca0b6 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:03.450097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cb0bd-0b8b-4215-9924-10535efca0b6 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:03.453840 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d0d19-7dc1-42c2-a20b-57b265d5c909 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:03.454470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d0d19-7dc1-42c2-a20b-57b265d5c909 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:03.454749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d0d19-7dc1-42c2-a20b-57b265d5c909 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:03.455023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d0d19-7dc1-42c2-a20b-57b265d5c909 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:03.456801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-194d0d19-7dc1-42c2-a20b-57b265d5c909 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:03.519689 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651a8d5-486b-42bc-9c93-a18d2186539d service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:03.543660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d0d19-7dc1-42c2-a20b-57b265d5c909 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:03.545203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d0d19-7dc1-42c2-a20b-57b265d5c909 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:03.548957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651a8d5-486b-42bc-9c93-a18d2186539d service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:03.552736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4dfd4-010d-4325-b158-73038c5ac12d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:03.553347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dfd4-010d-4325-b158-73038c5ac12d service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:03.553627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dfd4-010d-4325-b158-73038c5ac12d service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:03.553902 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dfd4-010d-4325-b158-73038c5ac12d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:03.554333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1240/2497] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:12:03.555896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-45f4dfd4-010d-4325-b158-73038c5ac12d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:03.616432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4dfd4-010d-4325-b158-73038c5ac12d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:03.617789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4dfd4-010d-4325-b158-73038c5ac12d service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:03.627171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1258/2498] 10.222.0.86 () {60 vars in 1396 bytes} [Wed Aug 24 10:12:03 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:12:03.725461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d0d19-7dc1-42c2-a20b-57b265d5c909 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:03.757283 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4dfd4-010d-4325-b158-73038c5ac12d service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:03.766142 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4dfd4-010d-4325-b158-73038c5ac12d service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:03.770519 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19eb2e-850a-4d30-b0ea-0aab30262aac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:03.771475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19eb2e-850a-4d30-b0ea-0aab30262aac service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:03.771813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19eb2e-850a-4d30-b0ea-0aab30262aac service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:03.772180 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19eb2e-850a-4d30-b0ea-0aab30262aac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:03.774066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fb19eb2e-850a-4d30-b0ea-0aab30262aac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:03.784222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d0d19-7dc1-42c2-a20b-57b265d5c909 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:03.795733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55bf027-42a5-49fc-b88e-e1d796b65b06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:03.796472 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bf027-42a5-49fc-b88e-e1d796b65b06 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:03.796759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bf027-42a5-49fc-b88e-e1d796b65b06 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:03.797036 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bf027-42a5-49fc-b88e-e1d796b65b06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:03.798861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d55bf027-42a5-49fc-b88e-e1d796b65b06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:03.843884 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19eb2e-850a-4d30-b0ea-0aab30262aac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:03.845478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19eb2e-850a-4d30-b0ea-0aab30262aac service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:03.856837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1259/2499] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:12:03.877662 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55bf027-42a5-49fc-b88e-e1d796b65b06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:03.879571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55bf027-42a5-49fc-b88e-e1d796b65b06 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:03.896238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1241/2500] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:12:11.026848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19eb2e-850a-4d30-b0ea-0aab30262aac service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:11.028290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fb19eb2e-850a-4d30-b0ea-0aab30262aac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:11.111316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:11.119498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19eb2e-850a-4d30-b0ea-0aab30262aac service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:11.125217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c8280-3b4f-47fa-9f6b-fd8c1116df93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:11.125805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c8280-3b4f-47fa-9f6b-fd8c1116df93 service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:11.126090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c8280-3b4f-47fa-9f6b-fd8c1116df93 service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:11.126366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c8280-3b4f-47fa-9f6b-fd8c1116df93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:11.128120 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0f3c8280-3b4f-47fa-9f6b-fd8c1116df93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:11.139414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c8280-3b4f-47fa-9f6b-fd8c1116df93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:11.141061 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c8280-3b4f-47fa-9f6b-fd8c1116df93 service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:11.150779 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1260/2501] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:12:13.889282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:21.146813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:03.990769 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55bf027-42a5-49fc-b88e-e1d796b65b06 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:04.000340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55bf027-42a5-49fc-b88e-e1d796b65b06 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:04.003868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edc828-605d-495d-b31a-f0eb278dcad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:04.004486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edc828-605d-495d-b31a-f0eb278dcad4 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:04.004759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edc828-605d-495d-b31a-f0eb278dcad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:04.005027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edc828-605d-495d-b31a-f0eb278dcad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:04.016419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edc828-605d-495d-b31a-f0eb278dcad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:04.017846 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edc828-605d-495d-b31a-f0eb278dcad4 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:04.029513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1242/2502] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:14:04.075501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c8280-3b4f-47fa-9f6b-fd8c1116df93 service placement] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:04.086826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c8280-3b4f-47fa-9f6b-fd8c1116df93 service placement] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:04.091951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261089f1-07ef-468b-a9e4-e635d94382fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:04.092629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261089f1-07ef-468b-a9e4-e635d94382fa service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:04.092911 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261089f1-07ef-468b-a9e4-e635d94382fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:04.093184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261089f1-07ef-468b-a9e4-e635d94382fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:04.104588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261089f1-07ef-468b-a9e4-e635d94382fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:04.106019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261089f1-07ef-468b-a9e4-e635d94382fa service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:04.116867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1261/2503] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:14:14.023996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:14.111959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:15.335303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edc828-605d-495d-b31a-f0eb278dcad4 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:15.354406 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edc828-605d-495d-b31a-f0eb278dcad4 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:15.358626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a22668-d7ff-450b-8c84-b59c41dd450d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:15.359225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a22668-d7ff-450b-8c84-b59c41dd450d service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:15.359534 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a22668-d7ff-450b-8c84-b59c41dd450d service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:15.359803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a22668-d7ff-450b-8c84-b59c41dd450d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:15.367144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a22668-d7ff-450b-8c84-b59c41dd450d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:15.368609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a22668-d7ff-450b-8c84-b59c41dd450d service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:15.376974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1243/2504] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:14:25.373812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=10 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:04.353209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261089f1-07ef-468b-a9e4-e635d94382fa service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:04.362070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261089f1-07ef-468b-a9e4-e635d94382fa service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:04.365777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5b949-81ab-44da-844b-fb5e3195f3e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:04.366385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5b949-81ab-44da-844b-fb5e3195f3e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:04.366664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5b949-81ab-44da-844b-fb5e3195f3e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:04.366938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5b949-81ab-44da-844b-fb5e3195f3e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:04.376985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5b949-81ab-44da-844b-fb5e3195f3e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:04.378583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5b949-81ab-44da-844b-fb5e3195f3e6 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:04.408463 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1262/2505] 10.222.0.86 () {60 vars in 1396 bytes} [Wed Aug 24 10:16:04 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:04.490841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a22668-d7ff-450b-8c84-b59c41dd450d service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:04.510181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a22668-d7ff-450b-8c84-b59c41dd450d service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:04.510181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c675f-52f0-4a32-9ee2-102e004e0f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:04.510181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c675f-52f0-4a32-9ee2-102e004e0f0f service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:04.510181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c675f-52f0-4a32-9ee2-102e004e0f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:04.511183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c675f-52f0-4a32-9ee2-102e004e0f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:04.518506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c675f-52f0-4a32-9ee2-102e004e0f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:04.520214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c675f-52f0-4a32-9ee2-102e004e0f0f service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:04.556933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1244/2506] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:04.639388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5b949-81ab-44da-844b-fb5e3195f3e6 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:04.654326 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5b949-81ab-44da-844b-fb5e3195f3e6 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:04.662205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791054c-57c6-4ff5-8467-a05b21533a1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:04.663089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791054c-57c6-4ff5-8467-a05b21533a1c service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:04.663567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791054c-57c6-4ff5-8467-a05b21533a1c service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:04.664038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791054c-57c6-4ff5-8467-a05b21533a1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:04.674048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791054c-57c6-4ff5-8467-a05b21533a1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:04.675641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791054c-57c6-4ff5-8467-a05b21533a1c service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:04.704787 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1263/2507] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:14.552198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=13 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:14.702075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=26 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:16.993017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c675f-52f0-4a32-9ee2-102e004e0f0f service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:16.995202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-373c675f-52f0-4a32-9ee2-102e004e0f0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:17.011224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c675f-52f0-4a32-9ee2-102e004e0f0f service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:17.016210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f81cd-3ae0-41a9-b339-e3259987a038 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:17.016814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f81cd-3ae0-41a9-b339-e3259987a038 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:17.017094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f81cd-3ae0-41a9-b339-e3259987a038 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:17.017400 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f81cd-3ae0-41a9-b339-e3259987a038 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:17.025880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f81cd-3ae0-41a9-b339-e3259987a038 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:17.027401 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f81cd-3ae0-41a9-b339-e3259987a038 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:17.037305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1245/2508] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:17:27.033086 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:03.648571 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791054c-57c6-4ff5-8467-a05b21533a1c service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:03.659068 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791054c-57c6-4ff5-8467-a05b21533a1c service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:03.662672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c5255-2ec0-47e4-a248-ef452006ae15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:03.663262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c5255-2ec0-47e4-a248-ef452006ae15 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:03.663542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c5255-2ec0-47e4-a248-ef452006ae15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:03.663813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c5255-2ec0-47e4-a248-ef452006ae15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:03.672355 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c5255-2ec0-47e4-a248-ef452006ae15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:03.673780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c5255-2ec0-47e4-a248-ef452006ae15 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:03.700807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1264/2509] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:04.774327 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f81cd-3ae0-41a9-b339-e3259987a038 service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:04.786362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f81cd-3ae0-41a9-b339-e3259987a038 service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:04.791578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a49da9-f95d-4983-af5a-40d143667d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:04.792237 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a49da9-f95d-4983-af5a-40d143667d3f service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:04.792596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a49da9-f95d-4983-af5a-40d143667d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:04.792799 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a49da9-f95d-4983-af5a-40d143667d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:04.803711 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a49da9-f95d-4983-af5a-40d143667d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:04.805196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a49da9-f95d-4983-af5a-40d143667d3f service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:04.833070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1246/2510] 10.222.0.86 () {60 vars in 1396 bytes} [Wed Aug 24 10:18:04 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:04.921197 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c5255-2ec0-47e4-a248-ef452006ae15 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:04.930217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c5255-2ec0-47e4-a248-ef452006ae15 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:04.933799 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae3929-fc2b-4900-b751-53cd7901fcf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:04.934386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3929-fc2b-4900-b751-53cd7901fcf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:04.934666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3929-fc2b-4900-b751-53cd7901fcf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:04.934938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3929-fc2b-4900-b751-53cd7901fcf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:04.943681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae3929-fc2b-4900-b751-53cd7901fcf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:04.945163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae3929-fc2b-4900-b751-53cd7901fcf5 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:04.972638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1265/2511] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:14.830225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=13 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:14.969693 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=26 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:21.617341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a49da9-f95d-4983-af5a-40d143667d3f service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:21.619603 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a1a49da9-f95d-4983-af5a-40d143667d3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:21.687752 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a49da9-f95d-4983-af5a-40d143667d3f service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:21.691399 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc402e7-ce2a-433c-b211-687dc1feeeef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:21.691994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc402e7-ce2a-433c-b211-687dc1feeeef service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:21.692337 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc402e7-ce2a-433c-b211-687dc1feeeef service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:21.692610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc402e7-ce2a-433c-b211-687dc1feeeef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:21.694441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4cc402e7-ce2a-433c-b211-687dc1feeeef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:21.704943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc402e7-ce2a-433c-b211-687dc1feeeef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:21.706371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc402e7-ce2a-433c-b211-687dc1feeeef service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:21.714283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1247/2512] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:19:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:19:31.710766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=31 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:05.048883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae3929-fc2b-4900-b751-53cd7901fcf5 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:05.050426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a6ae3929-fc2b-4900-b751-53cd7901fcf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:05.116329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae3929-fc2b-4900-b751-53cd7901fcf5 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:05.120260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5e4e8-222f-4da5-9c2e-c1a75c4fc88d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:05.120863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5e4e8-222f-4da5-9c2e-c1a75c4fc88d service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.121145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5e4e8-222f-4da5-9c2e-c1a75c4fc88d service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.121423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5e4e8-222f-4da5-9c2e-c1a75c4fc88d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.123325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9cd5e4e8-222f-4da5-9c2e-c1a75c4fc88d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:05.203985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5e4e8-222f-4da5-9c2e-c1a75c4fc88d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:05.205455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5e4e8-222f-4da5-9c2e-c1a75c4fc88d service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:05.217030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1266/2513] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:05.260114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc402e7-ce2a-433c-b211-687dc1feeeef service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:05.268206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc402e7-ce2a-433c-b211-687dc1feeeef service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:05.272009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6e521-81da-4da9-81de-8fe0a6029b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:05.272760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6e521-81da-4da9-81de-8fe0a6029b43 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.273820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6e521-81da-4da9-81de-8fe0a6029b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.274186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6e521-81da-4da9-81de-8fe0a6029b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.276152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-17b6e521-81da-4da9-81de-8fe0a6029b43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:05.290609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6e521-81da-4da9-81de-8fe0a6029b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:05.292125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6e521-81da-4da9-81de-8fe0a6029b43 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:05.303539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1248/2514] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:20:15.212119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=64 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:15.298489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:03.809578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5e4e8-222f-4da5-9c2e-c1a75c4fc88d service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:03.811142 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-9cd5e4e8-222f-4da5-9c2e-c1a75c4fc88d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:03.861801 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5e4e8-222f-4da5-9c2e-c1a75c4fc88d service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:03.866103 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf2963-64b0-4dab-a2ba-14f980bb2730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:03.866700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf2963-64b0-4dab-a2ba-14f980bb2730 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:03.866982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf2963-64b0-4dab-a2ba-14f980bb2730 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:03.867268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf2963-64b0-4dab-a2ba-14f980bb2730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:03.869072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f9cf2963-64b0-4dab-a2ba-14f980bb2730 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:03.952805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf2963-64b0-4dab-a2ba-14f980bb2730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:03.954250 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf2963-64b0-4dab-a2ba-14f980bb2730 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:03.964295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1267/2515] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:05.419986 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6e521-81da-4da9-81de-8fe0a6029b43 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:05.459474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6e521-81da-4da9-81de-8fe0a6029b43 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:05.464384 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c664c-bf9b-4e4b-a0f4-5d284947e3c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:05.464984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c664c-bf9b-4e4b-a0f4-5d284947e3c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:05.465261 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c664c-bf9b-4e4b-a0f4-5d284947e3c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:05.465531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c664c-bf9b-4e4b-a0f4-5d284947e3c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:05.467448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2d1c664c-bf9b-4e4b-a0f4-5d284947e3c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:05.546754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c664c-bf9b-4e4b-a0f4-5d284947e3c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:05.548238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c664c-bf9b-4e4b-a0f4-5d284947e3c8 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:05.557357 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1249/2516] 10.222.0.86 () {60 vars in 1396 bytes} [Wed Aug 24 10:22:05 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:05.646598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf2963-64b0-4dab-a2ba-14f980bb2730 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:05.655256 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf2963-64b0-4dab-a2ba-14f980bb2730 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:05.659091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce7682-d4ec-4c1e-8831-89a81754941d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:05.659747 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce7682-d4ec-4c1e-8831-89a81754941d service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:05.660034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce7682-d4ec-4c1e-8831-89a81754941d service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:05.660389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce7682-d4ec-4c1e-8831-89a81754941d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:05.662863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e1ce7682-d4ec-4c1e-8831-89a81754941d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:05.735076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce7682-d4ec-4c1e-8831-89a81754941d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:05.736570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce7682-d4ec-4c1e-8831-89a81754941d service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:05.745615 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1268/2517] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:15.553178 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=45 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:15.741442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:22.016360 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c664c-bf9b-4e4b-a0f4-5d284947e3c8 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:22.017997 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2d1c664c-bf9b-4e4b-a0f4-5d284947e3c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:22.116167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c664c-bf9b-4e4b-a0f4-5d284947e3c8 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:22.121112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adc154-ae48-4fb0-baa8-c80a17f40035 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:22.121706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc154-ae48-4fb0-baa8-c80a17f40035 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:22.121984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc154-ae48-4fb0-baa8-c80a17f40035 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:22.122255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc154-ae48-4fb0-baa8-c80a17f40035 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:22.124120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-92adc154-ae48-4fb0-baa8-c80a17f40035 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:22:22.135283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92adc154-ae48-4fb0-baa8-c80a17f40035 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:22.136900 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92adc154-ae48-4fb0-baa8-c80a17f40035 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:22.146869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1250/2518] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:32.142476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=44 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:04.121522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce7682-d4ec-4c1e-8831-89a81754941d service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:04.128756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce7682-d4ec-4c1e-8831-89a81754941d service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:04.129295 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adc154-ae48-4fb0-baa8-c80a17f40035 service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:04.130992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-92adc154-ae48-4fb0-baa8-c80a17f40035 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:04.132669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97b16d-cb0e-4bc6-8ec3-b0d8ba2ddf42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:04.133278 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97b16d-cb0e-4bc6-8ec3-b0d8ba2ddf42 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:04.133562 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97b16d-cb0e-4bc6-8ec3-b0d8ba2ddf42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:04.134059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97b16d-cb0e-4bc6-8ec3-b0d8ba2ddf42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.135986 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2a97b16d-cb0e-4bc6-8ec3-b0d8ba2ddf42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:04.170521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adc154-ae48-4fb0-baa8-c80a17f40035 service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:04.176546 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28871a3c-e673-41a1-ad84-17609bb63a91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:04.177147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28871a3c-e673-41a1-ad84-17609bb63a91 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:04.177428 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28871a3c-e673-41a1-ad84-17609bb63a91 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:04.177703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28871a3c-e673-41a1-ad84-17609bb63a91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.179572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-28871a3c-e673-41a1-ad84-17609bb63a91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:04.214928 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97b16d-cb0e-4bc6-8ec3-b0d8ba2ddf42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:04.216423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97b16d-cb0e-4bc6-8ec3-b0d8ba2ddf42 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:04.225424 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1269/2519] 10.222.0.86 () {60 vars in 1396 bytes} [Wed Aug 24 10:24:04 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:04.253019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28871a3c-e673-41a1-ad84-17609bb63a91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:04.254609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28871a3c-e673-41a1-ad84-17609bb63a91 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:04.264702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1251/2520] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:04.298976 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97b16d-cb0e-4bc6-8ec3-b0d8ba2ddf42 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:04.300870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2a97b16d-cb0e-4bc6-8ec3-b0d8ba2ddf42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:04.323047 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97b16d-cb0e-4bc6-8ec3-b0d8ba2ddf42 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:04.332849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5e899-c85d-49bb-ba2e-7ac56dd3235e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:04.333307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5e899-c85d-49bb-ba2e-7ac56dd3235e service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:04.333590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5e899-c85d-49bb-ba2e-7ac56dd3235e service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:04.334116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5e899-c85d-49bb-ba2e-7ac56dd3235e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.336610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-bbf5e899-c85d-49bb-ba2e-7ac56dd3235e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:04.428132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5e899-c85d-49bb-ba2e-7ac56dd3235e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:04.430250 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5e899-c85d-49bb-ba2e-7ac56dd3235e service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:04.442230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1270/2521] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:04.881040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28871a3c-e673-41a1-ad84-17609bb63a91 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:04.892016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28871a3c-e673-41a1-ad84-17609bb63a91 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:04.897585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a56275-0516-4b80-83f2-c3034b85d0dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:04.898055 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a56275-0516-4b80-83f2-c3034b85d0dd service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:04.898614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a56275-0516-4b80-83f2-c3034b85d0dd service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:04.898987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a56275-0516-4b80-83f2-c3034b85d0dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:04.902680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-96a56275-0516-4b80-83f2-c3034b85d0dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:04.990325 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a56275-0516-4b80-83f2-c3034b85d0dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:04.992126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a56275-0516-4b80-83f2-c3034b85d0dd service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:05.011029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1252/2522] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:07.854055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5e899-c85d-49bb-ba2e-7ac56dd3235e service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:07.865895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5e899-c85d-49bb-ba2e-7ac56dd3235e service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:07.871889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f47d5-a9f8-49ce-949e-db21c0679fe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:07.872523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f47d5-a9f8-49ce-949e-db21c0679fe0 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:07.872807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f47d5-a9f8-49ce-949e-db21c0679fe0 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:07.873090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f47d5-a9f8-49ce-949e-db21c0679fe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:07.874883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-869f47d5-a9f8-49ce-949e-db21c0679fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:07.889462 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f47d5-a9f8-49ce-949e-db21c0679fe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:07.892316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f47d5-a9f8-49ce-949e-db21c0679fe0 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:07.906616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1271/2523] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:09.138173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a56275-0516-4b80-83f2-c3034b85d0dd service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:09.160257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a56275-0516-4b80-83f2-c3034b85d0dd service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:09.177333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d2017-7f76-4f2f-aa0e-47acb966aa01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:09.177333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d2017-7f76-4f2f-aa0e-47acb966aa01 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:09.177333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d2017-7f76-4f2f-aa0e-47acb966aa01 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:09.177333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d2017-7f76-4f2f-aa0e-47acb966aa01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:09.177994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-178d2017-7f76-4f2f-aa0e-47acb966aa01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:09.194996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d2017-7f76-4f2f-aa0e-47acb966aa01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:09.196813 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d2017-7f76-4f2f-aa0e-47acb966aa01 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:09.210887 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1253/2524] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:17.900943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=72 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:19.204477 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=87 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:26.331264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f47d5-a9f8-49ce-949e-db21c0679fe0 service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:26.350518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f47d5-a9f8-49ce-949e-db21c0679fe0 service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:26.354185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a672fc-a0c7-49a0-9c61-5cdf2f89db3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:26.354777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a672fc-a0c7-49a0-9c61-5cdf2f89db3d service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:26.355059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a672fc-a0c7-49a0-9c61-5cdf2f89db3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:26.355333 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a672fc-a0c7-49a0-9c61-5cdf2f89db3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:26.362491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a672fc-a0c7-49a0-9c61-5cdf2f89db3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:26.363924 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a672fc-a0c7-49a0-9c61-5cdf2f89db3d service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:26.371806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1272/2525] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:36.368354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=10 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:05.242561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d2017-7f76-4f2f-aa0e-47acb966aa01 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:05.252207 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d2017-7f76-4f2f-aa0e-47acb966aa01 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:05.256395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce169e-52b5-43c0-b4dc-d143a02d811f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:05.256994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce169e-52b5-43c0-b4dc-d143a02d811f service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:05.257277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce169e-52b5-43c0-b4dc-d143a02d811f service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:05.257552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce169e-52b5-43c0-b4dc-d143a02d811f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:05.268632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce169e-52b5-43c0-b4dc-d143a02d811f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:05.270051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce169e-52b5-43c0-b4dc-d143a02d811f service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:05.281310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1254/2526] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:06.387231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a672fc-a0c7-49a0-9c61-5cdf2f89db3d service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:25:06.388823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f2a672fc-a0c7-49a0-9c61-5cdf2f89db3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:25:06.399868 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a672fc-a0c7-49a0-9c61-5cdf2f89db3d service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:25:06.403951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb432436-0bbd-4e91-af1b-db4bd17a1d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:25:06.404567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432436-0bbd-4e91-af1b-db4bd17a1d29 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:25:06.404849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432436-0bbd-4e91-af1b-db4bd17a1d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:25:06.405127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432436-0bbd-4e91-af1b-db4bd17a1d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:25:06.415202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb432436-0bbd-4e91-af1b-db4bd17a1d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:25:06.416673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb432436-0bbd-4e91-af1b-db4bd17a1d29 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:25:06.427993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1273/2527] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:25:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:25:15.276451 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:16.423197 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:03.801580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce169e-52b5-43c0-b4dc-d143a02d811f service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:03.813369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce169e-52b5-43c0-b4dc-d143a02d811f service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:03.818769 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671d781-8847-43ca-afb0-467bd293243e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:03.819363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671d781-8847-43ca-afb0-467bd293243e service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:03.819646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671d781-8847-43ca-afb0-467bd293243e service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:03.819924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671d781-8847-43ca-afb0-467bd293243e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:03.828007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671d781-8847-43ca-afb0-467bd293243e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:03.829601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671d781-8847-43ca-afb0-467bd293243e service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:03.855821 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1255/2528] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:27:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:27:04.822040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb432436-0bbd-4e91-af1b-db4bd17a1d29 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:04.835177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb432436-0bbd-4e91-af1b-db4bd17a1d29 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:04.841026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91943ad0-085e-4d21-bde9-4180e68904b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:04.841636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91943ad0-085e-4d21-bde9-4180e68904b7 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:04.841921 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91943ad0-085e-4d21-bde9-4180e68904b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:04.842195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91943ad0-085e-4d21-bde9-4180e68904b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:04.850632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91943ad0-085e-4d21-bde9-4180e68904b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:04.852260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91943ad0-085e-4d21-bde9-4180e68904b7 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:04.879728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1274/2529] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:27:04 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:27:13.853302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=13 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:14.877115 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=13 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:26.948964 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671d781-8847-43ca-afb0-467bd293243e service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:27:26.960432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671d781-8847-43ca-afb0-467bd293243e service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:27:26.965705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3325aea-0afb-49bb-a4c6-069b5c69e825 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:27:26.966305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325aea-0afb-49bb-a4c6-069b5c69e825 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:27:26.966587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325aea-0afb-49bb-a4c6-069b5c69e825 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:27:26.966857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325aea-0afb-49bb-a4c6-069b5c69e825 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:27:26.975380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3325aea-0afb-49bb-a4c6-069b5c69e825 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:27:26.976989 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3325aea-0afb-49bb-a4c6-069b5c69e825 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:27:26.988217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1256/2530] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:27:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:27:36.982616 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:06.345491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91943ad0-085e-4d21-bde9-4180e68904b7 service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:28:06.355712 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91943ad0-085e-4d21-bde9-4180e68904b7 service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:28:06.360269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d23ad-7180-4136-a7cf-70e5583f69e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:28:06.360879 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d23ad-7180-4136-a7cf-70e5583f69e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:28:06.361159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d23ad-7180-4136-a7cf-70e5583f69e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:28:06.361434 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d23ad-7180-4136-a7cf-70e5583f69e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:28:06.370314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d23ad-7180-4136-a7cf-70e5583f69e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:28:06.371792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d23ad-7180-4136-a7cf-70e5583f69e6 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:28:06.398153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1275/2531] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:28:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:28:16.395605 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=13 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:03.719878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3325aea-0afb-49bb-a4c6-069b5c69e825 service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:29:03.732520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3325aea-0afb-49bb-a4c6-069b5c69e825 service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:29:03.738139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933834a6-3159-4b61-8e37-ba20bb5b3576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:29:03.738726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933834a6-3159-4b61-8e37-ba20bb5b3576 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:29:03.739005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933834a6-3159-4b61-8e37-ba20bb5b3576 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:29:03.739277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933834a6-3159-4b61-8e37-ba20bb5b3576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:29:03.747631 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933834a6-3159-4b61-8e37-ba20bb5b3576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:29:03.749231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933834a6-3159-4b61-8e37-ba20bb5b3576 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:29:03.776147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1257/2532] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:29:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:29:13.773434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=13 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:32.310793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d23ad-7180-4136-a7cf-70e5583f69e6 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:29:32.312516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b53d23ad-7180-4136-a7cf-70e5583f69e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:29:32.382105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d23ad-7180-4136-a7cf-70e5583f69e6 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:29:32.385728 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501288bf-e77f-431b-99a6-4924a863dd19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:29:32.386324 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501288bf-e77f-431b-99a6-4924a863dd19 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:29:32.386598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501288bf-e77f-431b-99a6-4924a863dd19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:29:32.386866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501288bf-e77f-431b-99a6-4924a863dd19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:29:32.388697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-501288bf-e77f-431b-99a6-4924a863dd19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:29:32.397982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501288bf-e77f-431b-99a6-4924a863dd19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:29:32.399401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501288bf-e77f-431b-99a6-4924a863dd19 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:29:32.407675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1276/2533] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:29:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:29:42.404133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=31 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:05.788248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933834a6-3159-4b61-8e37-ba20bb5b3576 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:05.798484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933834a6-3159-4b61-8e37-ba20bb5b3576 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:05.802031 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b488e4-ed72-4b17-9828-4bed8a876f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:05.802618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b488e4-ed72-4b17-9828-4bed8a876f5e service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:05.802897 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b488e4-ed72-4b17-9828-4bed8a876f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:05.803168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b488e4-ed72-4b17-9828-4bed8a876f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:05.805605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-15b488e4-ed72-4b17-9828-4bed8a876f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:05.887463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b488e4-ed72-4b17-9828-4bed8a876f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:05.888943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b488e4-ed72-4b17-9828-4bed8a876f5e service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:05.899775 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1258/2534] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:30:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:30:06.816513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501288bf-e77f-431b-99a6-4924a863dd19 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:06.818134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b488e4-ed72-4b17-9828-4bed8a876f5e service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:06.820099 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-15b488e4-ed72-4b17-9828-4bed8a876f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:06.826111 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501288bf-e77f-431b-99a6-4924a863dd19 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:06.830315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf15942-8dc8-4a73-accd-68a9019054ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:06.830917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf15942-8dc8-4a73-accd-68a9019054ad service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:06.831226 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf15942-8dc8-4a73-accd-68a9019054ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:06.831498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf15942-8dc8-4a73-accd-68a9019054ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:06.833347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fcf15942-8dc8-4a73-accd-68a9019054ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:06.848106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf15942-8dc8-4a73-accd-68a9019054ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:06.849537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf15942-8dc8-4a73-accd-68a9019054ad service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:06.861236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1277/2535] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:30:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:30:06.892027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b488e4-ed72-4b17-9828-4bed8a876f5e service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:06.895315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bab52-c229-42c2-81ea-fe652a5d7352 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:06.895914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bab52-c229-42c2-81ea-fe652a5d7352 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:06.896222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bab52-c229-42c2-81ea-fe652a5d7352 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:06.896493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bab52-c229-42c2-81ea-fe652a5d7352 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:06.905687 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bab52-c229-42c2-81ea-fe652a5d7352 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:06.907115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bab52-c229-42c2-81ea-fe652a5d7352 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:06.916593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1259/2536] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:30:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:30:06.964661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf15942-8dc8-4a73-accd-68a9019054ad service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:06.973957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf15942-8dc8-4a73-accd-68a9019054ad service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:06.978205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029801e-29b9-4696-9d45-e15d347e53b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:06.978849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029801e-29b9-4696-9d45-e15d347e53b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:06.979132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029801e-29b9-4696-9d45-e15d347e53b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:06.979410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029801e-29b9-4696-9d45-e15d347e53b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:06.988106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029801e-29b9-4696-9d45-e15d347e53b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:06.989840 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029801e-29b9-4696-9d45-e15d347e53b9 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:07.016307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1278/2537] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:30:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:30:07.064169 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bab52-c229-42c2-81ea-fe652a5d7352 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:07.073443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bab52-c229-42c2-81ea-fe652a5d7352 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:07.077165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567e0f1-eeb6-473a-995e-7c31957d87d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:07.077756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567e0f1-eeb6-473a-995e-7c31957d87d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:07.078037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567e0f1-eeb6-473a-995e-7c31957d87d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:07.078314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567e0f1-eeb6-473a-995e-7c31957d87d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:07.087694 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567e0f1-eeb6-473a-995e-7c31957d87d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:07.089254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567e0f1-eeb6-473a-995e-7c31957d87d8 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:07.119078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1260/2538] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:30:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:30:09.423373 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029801e-29b9-4696-9d45-e15d347e53b9 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:09.434457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029801e-29b9-4696-9d45-e15d347e53b9 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:09.438377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ab5df-79d9-408d-9c7e-fdfd77c95ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:09.438987 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ab5df-79d9-408d-9c7e-fdfd77c95ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:09.439542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ab5df-79d9-408d-9c7e-fdfd77c95ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:09.439820 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ab5df-79d9-408d-9c7e-fdfd77c95ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:09.441756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d34ab5df-79d9-408d-9c7e-fdfd77c95ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:09.532625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ab5df-79d9-408d-9c7e-fdfd77c95ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:09.534238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ab5df-79d9-408d-9c7e-fdfd77c95ba4 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:09.554333 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1279/2539] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:30:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:30:10.148330 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567e0f1-eeb6-473a-995e-7c31957d87d8 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:10.158709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567e0f1-eeb6-473a-995e-7c31957d87d8 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:10.162554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d95a28-bff9-4ed7-b46d-a1075ce9b1c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:10.163159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d95a28-bff9-4ed7-b46d-a1075ce9b1c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:10.163626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d95a28-bff9-4ed7-b46d-a1075ce9b1c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:10.164124 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d95a28-bff9-4ed7-b46d-a1075ce9b1c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:10.166098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b4d95a28-bff9-4ed7-b46d-a1075ce9b1c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:10.178962 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d95a28-bff9-4ed7-b46d-a1075ce9b1c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:10.180487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d95a28-bff9-4ed7-b46d-a1075ce9b1c4 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:10.194563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1261/2540] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:30:14.019273 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ab5df-79d9-408d-9c7e-fdfd77c95ba4 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:14.021713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d34ab5df-79d9-408d-9c7e-fdfd77c95ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:14.052609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ab5df-79d9-408d-9c7e-fdfd77c95ba4 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:14.064804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39700687-fce9-408a-b915-98a553ce83dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:14.065697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39700687-fce9-408a-b915-98a553ce83dd service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:14.066148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39700687-fce9-408a-b915-98a553ce83dd service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:14.066624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39700687-fce9-408a-b915-98a553ce83dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:14.068660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-39700687-fce9-408a-b915-98a553ce83dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:14.093791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39700687-fce9-408a-b915-98a553ce83dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:14.101003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39700687-fce9-408a-b915-98a553ce83dd service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:14.159696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1280/2541] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:30:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:30:14.887199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d95a28-bff9-4ed7-b46d-a1075ce9b1c4 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:14.900004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d95a28-bff9-4ed7-b46d-a1075ce9b1c4 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:14.906044 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540b2fb-f427-45fa-8ef8-2b056d0a4416 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:14.906743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540b2fb-f427-45fa-8ef8-2b056d0a4416 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:14.907282 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540b2fb-f427-45fa-8ef8-2b056d0a4416 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:14.907636 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540b2fb-f427-45fa-8ef8-2b056d0a4416 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:14.909546 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9540b2fb-f427-45fa-8ef8-2b056d0a4416 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:14.923945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540b2fb-f427-45fa-8ef8-2b056d0a4416 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:14.925758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540b2fb-f427-45fa-8ef8-2b056d0a4416 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:14.937546 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1262/2542] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:30:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:30:19.470080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39700687-fce9-408a-b915-98a553ce83dd service placement] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:19.471590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-39700687-fce9-408a-b915-98a553ce83dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:19.497011 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39700687-fce9-408a-b915-98a553ce83dd service placement] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:19.514580 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:19.515770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:19.516149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:19.516493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] PATH_INFO: `/v3/users/1c665588c50d47dbaf1f8fde082ee73a` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:19.517827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c665588c50d47dbaf1f8fde082ee73a)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:19.519887 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:19.599947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c665588c50d47dbaf1f8fde082ee73a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:19.600640 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c665588c50d47dbaf1f8fde082ee73a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:19.602086 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c665588c50d47dbaf1f8fde082ee73a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:19.611196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c665588c50d47dbaf1f8fde082ee73a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:19.620628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c665588c50d47dbaf1f8fde082ee73a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:19.623116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:19.644285 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1281/2543] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:30:19 2022] DELETE /identity/v3/users/1c665588c50d47dbaf1f8fde082ee73a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:30:19.974538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540b2fb-f427-45fa-8ef8-2b056d0a4416 service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:19.988697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540b2fb-f427-45fa-8ef8-2b056d0a4416 service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:19.999798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:20.060229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:20.060895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:20.061374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:20.061506 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] PATH_INFO: `/v3/projects/302c5ab2e660436e8a0af25c12ceee20` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:20.062643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=302c5ab2e660436e8a0af25c12ceee20)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:20.064643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:20.118651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '302c5ab2e660436e8a0af25c12ceee20', 'request_id': 'req-4398e40c-830a-4d6c-a43c-6bb77ac887ed'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.126779 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '302c5ab2e660436e8a0af25c12ceee20', 'request_id': 'req-4398e40c-830a-4d6c-a43c-6bb77ac887ed'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.131846 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:20.133883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:20.137296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '302c5ab2e660436e8a0af25c12ceee20', 'request_id': 'req-4398e40c-830a-4d6c-a43c-6bb77ac887ed'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.150323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4c2fe-5cba-44ed-a6f8-12f1a4ce023e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:20.152753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] The token cache is being invalidate because project 302c5ab2e660436e8a0af25c12ceee20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:30:20.153047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.153806 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.159792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1263/2544] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:30:19 2022] DELETE /identity/v3/projects/302c5ab2e660436e8a0af25c12ceee20 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:30:20.161303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:20.163451 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:20.226694 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:20.227315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:20.227589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:20.227859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] PATH_INFO: `/v3/users/3ccfd2f7e35646b0a60b28b6807cee4b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:20.229027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ccfd2f7e35646b0a60b28b6807cee4b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:20.231088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:20.255705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398e40c-830a-4d6c-a43c-6bb77ac887ed admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:20.261967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:20.262730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:20.263302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:20.263559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] PATH_INFO: `/v3/users/4d2b8b636ffe44a0864f030c10bbfca9` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:20.279090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d2b8b636ffe44a0864f030c10bbfca9)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:20.280476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ccfd2f7e35646b0a60b28b6807cee4b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.281121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:20.281268 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ccfd2f7e35646b0a60b28b6807cee4b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.281854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ccfd2f7e35646b0a60b28b6807cee4b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.290026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ccfd2f7e35646b0a60b28b6807cee4b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.298317 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ccfd2f7e35646b0a60b28b6807cee4b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.316336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:20.317566 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1282/2545] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:30:20 2022] DELETE /identity/v3/users/3ccfd2f7e35646b0a60b28b6807cee4b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:30:20.331477 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d2b8b636ffe44a0864f030c10bbfca9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.332196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d2b8b636ffe44a0864f030c10bbfca9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.332856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d2b8b636ffe44a0864f030c10bbfca9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.341585 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d2b8b636ffe44a0864f030c10bbfca9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.349887 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d2b8b636ffe44a0864f030c10bbfca9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.371311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1264/2546] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:30:20 2022] DELETE /identity/v3/users/4d2b8b636ffe44a0864f030c10bbfca9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:30:20.500838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:20.502856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:20.560816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:20.562412 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:20.604361 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186aaecf-d642-4d73-9e39-a551d142cfc0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:20.609151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:20.635593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392ce9c-d5c9-4835-8b04-c7bcce22b0d9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:20.640369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:20.641066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:20.641434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:20.641729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] PATH_INFO: `/v3/projects/1812511b96a64c2f95b2e66892644cec` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:20.652422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1812511b96a64c2f95b2e66892644cec)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:20.654459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:20.665176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:20.665778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:20.666234 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:20.666353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] PATH_INFO: `/v3/projects/dc33f347b062491aaa04b7dc18553cf4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:20.667537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc33f347b062491aaa04b7dc18553cf4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:20.669776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:20.706525 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1812511b96a64c2f95b2e66892644cec', 'request_id': 'req-fdaf3700-ac22-4baf-99c3-59b67832d9a8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.713501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1812511b96a64c2f95b2e66892644cec', 'request_id': 'req-fdaf3700-ac22-4baf-99c3-59b67832d9a8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.718016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc33f347b062491aaa04b7dc18553cf4', 'request_id': 'req-519549a2-eadc-4cd6-ad28-f4b6a670cfae'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.719702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1812511b96a64c2f95b2e66892644cec', 'request_id': 'req-fdaf3700-ac22-4baf-99c3-59b67832d9a8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.724541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc33f347b062491aaa04b7dc18553cf4', 'request_id': 'req-519549a2-eadc-4cd6-ad28-f4b6a670cfae'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.731518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc33f347b062491aaa04b7dc18553cf4', 'request_id': 'req-519549a2-eadc-4cd6-ad28-f4b6a670cfae'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.733861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] The token cache is being invalidate because project 1812511b96a64c2f95b2e66892644cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:30:20.734257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.735146 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.736698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1265/2547] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:30:20 2022] DELETE /identity/v3/projects/1812511b96a64c2f95b2e66892644cec => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:30:20.745489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] The token cache is being invalidate because project dc33f347b062491aaa04b7dc18553cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:30:20.745885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.746641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.748311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1283/2548] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:30:20 2022] DELETE /identity/v3/projects/dc33f347b062491aaa04b7dc18553cf4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:30:20.752456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:20.753909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:20.798675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:20.835764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf3700-ac22-4baf-99c3-59b67832d9a8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:20.903638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:20.904322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:20.904605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:20.904876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] PATH_INFO: `/v3/users/147ad0a9182d40128b64ecda50abd8a0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:20.921302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=147ad0a9182d40128b64ecda50abd8a0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:20.923258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:20.982050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '147ad0a9182d40128b64ecda50abd8a0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.984145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '147ad0a9182d40128b64ecda50abd8a0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.985539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '147ad0a9182d40128b64ecda50abd8a0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:20.996830 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '147ad0a9182d40128b64ecda50abd8a0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:21.005756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '147ad0a9182d40128b64ecda50abd8a0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:21.028188 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1266/2549] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:30:20 2022] DELETE /identity/v3/users/147ad0a9182d40128b64ecda50abd8a0 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:30:21.208750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:30:21.210718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:30:21.224407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519549a2-eadc-4cd6-ad28-f4b6a670cfae admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:30:21.283716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:30:21.284419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:30:21.284700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:30:21.284975 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] PATH_INFO: `/v3/projects/ad806acbc6e84af6a46ddf2c13eb798f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:30:21.296560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad806acbc6e84af6a46ddf2c13eb798f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:30:21.298513 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:30:21.340062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:21.355071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad806acbc6e84af6a46ddf2c13eb798f', 'request_id': 'req-58f5ed8c-7f42-472a-87d2-147ef7daadf8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:21.362782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad806acbc6e84af6a46ddf2c13eb798f', 'request_id': 'req-58f5ed8c-7f42-472a-87d2-147ef7daadf8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:21.369856 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad806acbc6e84af6a46ddf2c13eb798f', 'request_id': 'req-58f5ed8c-7f42-472a-87d2-147ef7daadf8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:21.385062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] The token cache is being invalidate because project ad806acbc6e84af6a46ddf2c13eb798f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:30:21.385263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:21.385917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:30:21.387455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1284/2550] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:30:21 2022] DELETE /identity/v3/projects/ad806acbc6e84af6a46ddf2c13eb798f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:30:31.023947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=68,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:31.381007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:07.490351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:32:07.491953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:32:07.590604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975af3c8-ad22-4b86-a8ba-0c01e916e4cd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:32:07.653376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224981d-3672-4a65-b2ed-4f43975c9092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:32:07.653985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224981d-3672-4a65-b2ed-4f43975c9092 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:32:07.654265 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224981d-3672-4a65-b2ed-4f43975c9092 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:32:07.654539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224981d-3672-4a65-b2ed-4f43975c9092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:32:07.656349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9224981d-3672-4a65-b2ed-4f43975c9092 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:32:07.742123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224981d-3672-4a65-b2ed-4f43975c9092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:32:07.743481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224981d-3672-4a65-b2ed-4f43975c9092 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:32:07.752335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1267/2551] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:32:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:32:07.800836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:32:07.802354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:32:07.818184 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5ed8c-7f42-472a-87d2-147ef7daadf8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:32:07.821918 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3c0d5-3cc3-4ccd-bd96-d1ffa2f96e8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:32:07.822512 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3c0d5-3cc3-4ccd-bd96-d1ffa2f96e8b service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:32:07.822794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3c0d5-3cc3-4ccd-bd96-d1ffa2f96e8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:32:07.823071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3c0d5-3cc3-4ccd-bd96-d1ffa2f96e8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:32:07.824826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dbd3c0d5-3cc3-4ccd-bd96-d1ffa2f96e8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:32:07.894790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3c0d5-3cc3-4ccd-bd96-d1ffa2f96e8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:32:07.896257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3c0d5-3cc3-4ccd-bd96-d1ffa2f96e8b service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:32:07.904989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1285/2552] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:32:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:32:09.619358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224981d-3672-4a65-b2ed-4f43975c9092 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:32:09.620919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9224981d-3672-4a65-b2ed-4f43975c9092 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:32:09.638304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:09.713870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224981d-3672-4a65-b2ed-4f43975c9092 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:32:09.719481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326ffec-6a63-4107-867d-b3fbfce1fff3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:32:09.720284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ffec-6a63-4107-867d-b3fbfce1fff3 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:32:09.720578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ffec-6a63-4107-867d-b3fbfce1fff3 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:32:09.720854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ffec-6a63-4107-867d-b3fbfce1fff3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:32:09.722653 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f326ffec-6a63-4107-867d-b3fbfce1fff3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:32:09.736007 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326ffec-6a63-4107-867d-b3fbfce1fff3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:32:09.737730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326ffec-6a63-4107-867d-b3fbfce1fff3 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:32:09.750298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1268/2553] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:32:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:32:17.901038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=35 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:19.744575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=30 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:34.914654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3c0d5-3cc3-4ccd-bd96-d1ffa2f96e8b service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:32:34.916088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dbd3c0d5-3cc3-4ccd-bd96-d1ffa2f96e8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:32:35.013437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3c0d5-3cc3-4ccd-bd96-d1ffa2f96e8b service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:32:35.018608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d551908-e917-4b23-8c73-200bdcca00e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:32:35.019207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d551908-e917-4b23-8c73-200bdcca00e2 service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:32:35.019487 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d551908-e917-4b23-8c73-200bdcca00e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:32:35.019760 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d551908-e917-4b23-8c73-200bdcca00e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:32:35.021551 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6d551908-e917-4b23-8c73-200bdcca00e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:32:35.032866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d551908-e917-4b23-8c73-200bdcca00e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:32:35.034461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d551908-e917-4b23-8c73-200bdcca00e2 service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:32:35.044140 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1286/2554] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:32:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:32:45.040008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=44 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:07.464203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326ffec-6a63-4107-867d-b3fbfce1fff3 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:33:07.465703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f326ffec-6a63-4107-867d-b3fbfce1fff3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:33:07.493838 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326ffec-6a63-4107-867d-b3fbfce1fff3 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:33:07.500834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030879e-138e-4bd8-95de-3fc450c1e014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:33:07.501634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030879e-138e-4bd8-95de-3fc450c1e014 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:33:07.501945 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030879e-138e-4bd8-95de-3fc450c1e014 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:33:07.502349 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030879e-138e-4bd8-95de-3fc450c1e014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:33:07.504212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e030879e-138e-4bd8-95de-3fc450c1e014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:33:07.576263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030879e-138e-4bd8-95de-3fc450c1e014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:33:07.577921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030879e-138e-4bd8-95de-3fc450c1e014 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:33:07.587443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1269/2555] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:33:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:33:17.583674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=36 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:08.241687 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d551908-e917-4b23-8c73-200bdcca00e2 service placement] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:34:08.250901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d551908-e917-4b23-8c73-200bdcca00e2 service placement] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:34:08.255674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2f683-d258-4da2-bce5-a0e78874a0ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:34:08.256305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2f683-d258-4da2-bce5-a0e78874a0ab service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:34:08.256564 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2f683-d258-4da2-bce5-a0e78874a0ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:34:08.256831 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2f683-d258-4da2-bce5-a0e78874a0ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:34:08.258937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-19d2f683-d258-4da2-bce5-a0e78874a0ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application [None req-19d2f683-d258-4da2-bce5-a0e78874a0ab service cinder] Could not find project: 302c5ab2e660436e8a0af25c12ceee20.: keystone.exception.ProjectNotFound: Could not find project: 302c5ab2e660436e8a0af25c12ceee20. Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 24 10:34:08.292407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:34:08.293717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:34:08.295009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application with Lock( Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:34:08.296322 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:34:08.297536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 302c5ab2e660436e8a0af25c12ceee20. Aug 24 10:34:08.297536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: ERROR keystone.server.flask.application Aug 24 10:34:08.297536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1287/2556] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:34:08 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 53 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 24 10:34:08.736072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77be42-7339-4614-bec9-479b582d79f2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:34:08.737182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77be42-7339-4614-bec9-479b582d79f2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:34:08.737290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77be42-7339-4614-bec9-479b582d79f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:34:08.771583 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7c77be42-7339-4614-bec9-479b582d79f2 None None] MFA Rules not processed for user `df9b402589fa4020bc241a4b9c06dd91`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:34:08.773946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7c77be42-7339-4614-bec9-479b582d79f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:34:08.781335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1270/2557] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:34:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:34:09.096749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2f683-d258-4da2-bce5-a0e78874a0ab service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:34:09.098164 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-19d2f683-d258-4da2-bce5-a0e78874a0ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:34:09.154389 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2f683-d258-4da2-bce5-a0e78874a0ab service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:34:09.157972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b6266-3eb1-4a24-a3ef-529f97f576e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:34:09.158622 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b6266-3eb1-4a24-a3ef-529f97f576e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:34:09.158910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b6266-3eb1-4a24-a3ef-529f97f576e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:34:09.159185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b6266-3eb1-4a24-a3ef-529f97f576e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:34:09.160895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a25b6266-3eb1-4a24-a3ef-529f97f576e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:34:09.170099 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b6266-3eb1-4a24-a3ef-529f97f576e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:34:09.171523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b6266-3eb1-4a24-a3ef-529f97f576e9 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:34:09.179516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1288/2558] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:34:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:34:09.523800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77be42-7339-4614-bec9-479b582d79f2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:34:09.536939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77be42-7339-4614-bec9-479b582d79f2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:34:09.542108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b134abb-0f10-4362-9d8a-63b48e44c7b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:34:09.542791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b134abb-0f10-4362-9d8a-63b48e44c7b3 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:34:09.543078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b134abb-0f10-4362-9d8a-63b48e44c7b3 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:34:09.543385 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b134abb-0f10-4362-9d8a-63b48e44c7b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:34:09.545295 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9b134abb-0f10-4362-9d8a-63b48e44c7b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:34:09.559235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b134abb-0f10-4362-9d8a-63b48e44c7b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:34:09.561298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b134abb-0f10-4362-9d8a-63b48e44c7b3 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:34:09.576407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1271/2559] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:34:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:34:18.239412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b6266-3eb1-4a24-a3ef-529f97f576e9 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:34:18.273048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b6266-3eb1-4a24-a3ef-529f97f576e9 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:34:18.280067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79bd6-0c76-496a-a5f0-cbccc259f1bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:34:18.281016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79bd6-0c76-496a-a5f0-cbccc259f1bf service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:34:18.281710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79bd6-0c76-496a-a5f0-cbccc259f1bf service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:34:18.282206 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79bd6-0c76-496a-a5f0-cbccc259f1bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:34:18.284535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8ed79bd6-0c76-496a-a5f0-cbccc259f1bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:34:18.301057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed79bd6-0c76-496a-a5f0-cbccc259f1bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:34:18.303518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed79bd6-0c76-496a-a5f0-cbccc259f1bf service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:34:18.315030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1289/2560] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:34:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:34:19.572313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=12 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:28.310127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=43 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:05.828717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b134abb-0f10-4362-9d8a-63b48e44c7b3 service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:35:05.848277 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b134abb-0f10-4362-9d8a-63b48e44c7b3 service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:35:05.851941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec3af6-2600-4be3-8dd3-f74b146ae03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:35:05.852561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec3af6-2600-4be3-8dd3-f74b146ae03d service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:35:05.852842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec3af6-2600-4be3-8dd3-f74b146ae03d service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:35:05.853118 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec3af6-2600-4be3-8dd3-f74b146ae03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:35:05.854869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-24ec3af6-2600-4be3-8dd3-f74b146ae03d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:35:05.869013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec3af6-2600-4be3-8dd3-f74b146ae03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:35:05.870508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec3af6-2600-4be3-8dd3-f74b146ae03d service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:35:05.881668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1272/2561] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:35:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:35:07.011525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79bd6-0c76-496a-a5f0-cbccc259f1bf service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:35:07.022620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79bd6-0c76-496a-a5f0-cbccc259f1bf service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:35:07.028766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc940-aca5-4e0c-9c6d-c0c306e405e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:35:07.029357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc940-aca5-4e0c-9c6d-c0c306e405e7 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:35:07.029641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc940-aca5-4e0c-9c6d-c0c306e405e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:35:07.029915 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc940-aca5-4e0c-9c6d-c0c306e405e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:35:07.031626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5e4bc940-aca5-4e0c-9c6d-c0c306e405e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:35:07.101927 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bc940-aca5-4e0c-9c6d-c0c306e405e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:35:07.103531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bc940-aca5-4e0c-9c6d-c0c306e405e7 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:35:07.112413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1290/2562] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:35:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:35:07.162610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec3af6-2600-4be3-8dd3-f74b146ae03d service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:35:07.164013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-24ec3af6-2600-4be3-8dd3-f74b146ae03d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:35:07.175487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec3af6-2600-4be3-8dd3-f74b146ae03d service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:35:07.179167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbea80-05da-4d9a-b9e5-2e63d95633d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:35:07.179789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbea80-05da-4d9a-b9e5-2e63d95633d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:35:07.180090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbea80-05da-4d9a-b9e5-2e63d95633d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:35:07.180368 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbea80-05da-4d9a-b9e5-2e63d95633d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:35:07.182050 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-34dbea80-05da-4d9a-b9e5-2e63d95633d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:35:07.195523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbea80-05da-4d9a-b9e5-2e63d95633d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:35:07.196980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbea80-05da-4d9a-b9e5-2e63d95633d1 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:35:07.207790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1273/2563] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:35:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:35:07.674557 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc940-aca5-4e0c-9c6d-c0c306e405e7 service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:35:07.686211 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc940-aca5-4e0c-9c6d-c0c306e405e7 service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:35:07.691616 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d212f70-dbef-40a4-9553-532c1129c682 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:35:07.692264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d212f70-dbef-40a4-9553-532c1129c682 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:35:07.692549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d212f70-dbef-40a4-9553-532c1129c682 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:35:07.692824 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d212f70-dbef-40a4-9553-532c1129c682 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:35:07.694590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4d212f70-dbef-40a4-9553-532c1129c682 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:35:07.775149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d212f70-dbef-40a4-9553-532c1129c682 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:35:07.776809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d212f70-dbef-40a4-9553-532c1129c682 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:35:07.797634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1291/2564] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:35:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:35:17.202966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=16 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:17.795689 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=74 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:38.347739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbea80-05da-4d9a-b9e5-2e63d95633d1 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:35:38.356533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbea80-05da-4d9a-b9e5-2e63d95633d1 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:35:38.361346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c215e43-cc3b-4189-b1c3-a2cc122ec0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:35:38.362181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c215e43-cc3b-4189-b1c3-a2cc122ec0fc service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:35:38.362459 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c215e43-cc3b-4189-b1c3-a2cc122ec0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:35:38.362726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c215e43-cc3b-4189-b1c3-a2cc122ec0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:35:38.364491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1c215e43-cc3b-4189-b1c3-a2cc122ec0fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:35:38.374172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c215e43-cc3b-4189-b1c3-a2cc122ec0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:35:38.375596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c215e43-cc3b-4189-b1c3-a2cc122ec0fc service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:35:38.384393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1274/2565] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:35:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:35:48.380899 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:10.836935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4a6c-e134-4127-bb15-3f1c6aa3ae21 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:36:10.837458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4a6c-e134-4127-bb15-3f1c6aa3ae21 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:36:10.837458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4a6c-e134-4127-bb15-3f1c6aa3ae21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:36:10.912207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2f9e4a6c-e134-4127-bb15-3f1c6aa3ae21 None None] MFA Rules not processed for user `1bac9dc42db9451e9e87a88c17f29c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:36:10.913442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2f9e4a6c-e134-4127-bb15-3f1c6aa3ae21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:36:10.991282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1292/2566] 10.222.0.86 () {58 vars in 967 bytes} [Wed Aug 24 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:36:11.046328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c215e43-cc3b-4189-b1c3-a2cc122ec0fc service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:36:11.047842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1c215e43-cc3b-4189-b1c3-a2cc122ec0fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:36:11.143572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c215e43-cc3b-4189-b1c3-a2cc122ec0fc service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:36:11.149387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d38dd1-9300-4111-96d0-84aff70c0dbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1b1d85195db46b19d6a0156dbdf7f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:36:11.150029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d38dd1-9300-4111-96d0-84aff70c0dbe service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:36:11.150307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d38dd1-9300-4111-96d0-84aff70c0dbe service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:36:11.150584 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d38dd1-9300-4111-96d0-84aff70c0dbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:36:11.159078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d38dd1-9300-4111-96d0-84aff70c0dbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:36:11.160726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d38dd1-9300-4111-96d0-84aff70c0dbe service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:36:11.170001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1275/2567] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Aug 24 10:36:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:36:11.668009 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e4a6c-e134-4127-bb15-3f1c6aa3ae21 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:36:11.678732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e4a6c-e134-4127-bb15-3f1c6aa3ae21 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:36:11.682901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2340136-0663-4765-af15-8ddbec50f3fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:36:11.683505 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2340136-0663-4765-af15-8ddbec50f3fc service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:36:11.683783 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2340136-0663-4765-af15-8ddbec50f3fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:36:11.684072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2340136-0663-4765-af15-8ddbec50f3fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:36:11.685970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a2340136-0663-4765-af15-8ddbec50f3fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:36:11.752672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2340136-0663-4765-af15-8ddbec50f3fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:36:11.754133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2340136-0663-4765-af15-8ddbec50f3fc service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:36:11.765611 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1293/2568] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:36:21.166441 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=41 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:21.760483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=74 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:07.823629 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d38dd1-9300-4111-96d0-84aff70c0dbe service swift] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:37:07.835568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d38dd1-9300-4111-96d0-84aff70c0dbe service swift] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:37:07.841056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e81059-5aec-474d-8353-ea7328e1bc9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:37:07.841645 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e81059-5aec-474d-8353-ea7328e1bc9a service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:37:07.841929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e81059-5aec-474d-8353-ea7328e1bc9a service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:37:07.842212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e81059-5aec-474d-8353-ea7328e1bc9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:37:07.850569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e81059-5aec-474d-8353-ea7328e1bc9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:37:07.852196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e81059-5aec-474d-8353-ea7328e1bc9a service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:37:07.880587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1276/2569] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:37:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:37:08.072295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2340136-0663-4765-af15-8ddbec50f3fc service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:37:08.084483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2340136-0663-4765-af15-8ddbec50f3fc service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:37:08.089752 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e95e8-603d-4761-a76b-f7bef0591963 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:37:08.090365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e95e8-603d-4761-a76b-f7bef0591963 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:37:08.090645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e95e8-603d-4761-a76b-f7bef0591963 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:37:08.090916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e95e8-603d-4761-a76b-f7bef0591963 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:37:08.099262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e95e8-603d-4761-a76b-f7bef0591963 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:37:08.100908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e95e8-603d-4761-a76b-f7bef0591963 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:37:08.128421 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1294/2570] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:37:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:37:17.877784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=13 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:18.125697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=13 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:36.960476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e81059-5aec-474d-8353-ea7328e1bc9a service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:37:36.961897 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-34e81059-5aec-474d-8353-ea7328e1bc9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:37:36.978923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e81059-5aec-474d-8353-ea7328e1bc9a service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:37:36.984470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a07ba6-c7f5-4b0f-9ca6-b417496b154f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:37:36.985078 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07ba6-c7f5-4b0f-9ca6-b417496b154f service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:37:36.985361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07ba6-c7f5-4b0f-9ca6-b417496b154f service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:37:36.985640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07ba6-c7f5-4b0f-9ca6-b417496b154f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:37:36.994576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a07ba6-c7f5-4b0f-9ca6-b417496b154f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:37:36.996361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a07ba6-c7f5-4b0f-9ca6-b417496b154f service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:37:37.007205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1277/2571] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:37:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:37:47.002192 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:07.049895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e95e8-603d-4761-a76b-f7bef0591963 service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:07.064274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e95e8-603d-4761-a76b-f7bef0591963 service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:07.070437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb3438-4e97-4a78-8d9c-415e36521c87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:07.071075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb3438-4e97-4a78-8d9c-415e36521c87 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:07.071358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb3438-4e97-4a78-8d9c-415e36521c87 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:07.071635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb3438-4e97-4a78-8d9c-415e36521c87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:07.082710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb3438-4e97-4a78-8d9c-415e36521c87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:07.085422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb3438-4e97-4a78-8d9c-415e36521c87 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:07.100106 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1295/2572] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:08.014649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a07ba6-c7f5-4b0f-9ca6-b417496b154f service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:08.017249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-07a07ba6-c7f5-4b0f-9ca6-b417496b154f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:08.032691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a07ba6-c7f5-4b0f-9ca6-b417496b154f service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:08.036826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ae57d-d1cd-4731-84d9-062ec53a1146 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:08.037519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ae57d-d1cd-4731-84d9-062ec53a1146 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:08.038014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ae57d-d1cd-4731-84d9-062ec53a1146 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:08.038583 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ae57d-d1cd-4731-84d9-062ec53a1146 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:08.050784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ae57d-d1cd-4731-84d9-062ec53a1146 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:08.052296 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ae57d-d1cd-4731-84d9-062ec53a1146 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:08.084956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1278/2573] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:10.176469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04443ef-2d88-4982-bfcd-b40b03b9b5fc None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:10.177136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04443ef-2d88-4982-bfcd-b40b03b9b5fc None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:10.177136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04443ef-2d88-4982-bfcd-b40b03b9b5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:10.210474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a04443ef-2d88-4982-bfcd-b40b03b9b5fc None None] MFA Rules not processed for user `1de97cce55e944dbb62a920ea11734f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:10.211672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a04443ef-2d88-4982-bfcd-b40b03b9b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:10.219451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1296/2574] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:18.082102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=13 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:18.712666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075b26-357e-4d5a-b7c9-d356203a8a43 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:18.712949 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075b26-357e-4d5a-b7c9-d356203a8a43 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:18.713224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075b26-357e-4d5a-b7c9-d356203a8a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:18.743916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b8075b26-357e-4d5a-b7c9-d356203a8a43 None None] MFA Rules not processed for user `4ce9eb7417a7482cb9cdaed16cb6aeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:18.745249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b8075b26-357e-4d5a-b7c9-d356203a8a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:18.752495 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1279/2575] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:18.993549 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04443ef-2d88-4982-bfcd-b40b03b9b5fc None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:19.003231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04443ef-2d88-4982-bfcd-b40b03b9b5fc None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:19.007228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950f1cc-1ff9-410c-aaca-4ca82850ba5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:19.008536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950f1cc-1ff9-410c-aaca-4ca82850ba5b service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:19.008536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950f1cc-1ff9-410c-aaca-4ca82850ba5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:19.008536 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950f1cc-1ff9-410c-aaca-4ca82850ba5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:19.010633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4950f1cc-1ff9-410c-aaca-4ca82850ba5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:19.021789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950f1cc-1ff9-410c-aaca-4ca82850ba5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:19.023526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950f1cc-1ff9-410c-aaca-4ca82850ba5b service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:19.032898 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1297/2576] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:24.426713 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8075b26-357e-4d5a-b7c9-d356203a8a43 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:24.437725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8075b26-357e-4d5a-b7c9-d356203a8a43 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:24.441974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05f8eb-2471-46ce-90d7-f119b9591ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:24.442633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05f8eb-2471-46ce-90d7-f119b9591ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:24.442912 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05f8eb-2471-46ce-90d7-f119b9591ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:24.443369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05f8eb-2471-46ce-90d7-f119b9591ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:24.445199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ba05f8eb-2471-46ce-90d7-f119b9591ff2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:24.459329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05f8eb-2471-46ce-90d7-f119b9591ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:24.460898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05f8eb-2471-46ce-90d7-f119b9591ff2 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:24.472715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1280/2577] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:38:27.957832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950f1cc-1ff9-410c-aaca-4ca82850ba5b service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:27.973631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950f1cc-1ff9-410c-aaca-4ca82850ba5b service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:27.992790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4631f2-ed0d-401e-b6ae-f3371e93d4bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:27.993787 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4631f2-ed0d-401e-b6ae-f3371e93d4bb service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:27.994802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4631f2-ed0d-401e-b6ae-f3371e93d4bb service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:27.995441 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4631f2-ed0d-401e-b6ae-f3371e93d4bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:27.997669 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6a4631f2-ed0d-401e-b6ae-f3371e93d4bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:28.019122 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4631f2-ed0d-401e-b6ae-f3371e93d4bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:28.021266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4631f2-ed0d-401e-b6ae-f3371e93d4bb service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:28.035435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1298/2578] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:28.054575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05f8eb-2471-46ce-90d7-f119b9591ff2 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:28.056005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ba05f8eb-2471-46ce-90d7-f119b9591ff2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:28.081723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05f8eb-2471-46ce-90d7-f119b9591ff2 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:28.086491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8a298-b26d-4035-ab01-9c106667fd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:28.087129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8a298-b26d-4035-ab01-9c106667fd95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:28.087412 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8a298-b26d-4035-ab01-9c106667fd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:28.087684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8a298-b26d-4035-ab01-9c106667fd95 admin admin] PATH_INFO: `/v3/users/46c01f8b99e24be5b6c2d73b9b1bd9b8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:28.088957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8a298-b26d-4035-ab01-9c106667fd95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c01f8b99e24be5b6c2d73b9b1bd9b8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:28.090876 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8a298-b26d-4035-ab01-9c106667fd95 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:28.156376 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-42b8a298-b26d-4035-ab01-9c106667fd95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c01f8b99e24be5b6c2d73b9b1bd9b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:28.157177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-42b8a298-b26d-4035-ab01-9c106667fd95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c01f8b99e24be5b6c2d73b9b1bd9b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:28.158292 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-42b8a298-b26d-4035-ab01-9c106667fd95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c01f8b99e24be5b6c2d73b9b1bd9b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:28.172853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-42b8a298-b26d-4035-ab01-9c106667fd95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c01f8b99e24be5b6c2d73b9b1bd9b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:28.192675 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-42b8a298-b26d-4035-ab01-9c106667fd95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c01f8b99e24be5b6c2d73b9b1bd9b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:28.226751 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1281/2579] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:38:28 2022] DELETE /identity/v3/users/46c01f8b99e24be5b6c2d73b9b1bd9b8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:28.499069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4631f2-ed0d-401e-b6ae-f3371e93d4bb service placement] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:28.504888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=25 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:28.517756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4631f2-ed0d-401e-b6ae-f3371e93d4bb service placement] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:28.611744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:28.612413 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:28.612721 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:28.613004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] PATH_INFO: `/v3/projects/1bdfd8ab42f9496d87a8cfcf78d7edc3` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:28.614143 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bdfd8ab42f9496d87a8cfcf78d7edc3)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:28.616115 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:28.666576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'request_id': 'req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:28.672903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'request_id': 'req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:28.678865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'request_id': 'req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:28.692875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] The token cache is being invalidate because project 1bdfd8ab42f9496d87a8cfcf78d7edc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:38:28.693166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:28.693936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-a76ec3d1-7931-4fc1-92c7-b0c68adb7d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:28.695469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1299/2580] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:38:28 2022] DELETE /identity/v3/projects/1bdfd8ab42f9496d87a8cfcf78d7edc3 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:28.700565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac40da-e106-40a7-89d7-7f62f3088deb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:28.700853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac40da-e106-40a7-89d7-7f62f3088deb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:28.701127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac40da-e106-40a7-89d7-7f62f3088deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:28.760837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b3ac40da-e106-40a7-89d7-7f62f3088deb None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:28.762051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b3ac40da-e106-40a7-89d7-7f62f3088deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:28.828216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1282/2581] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:38:28.836385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdf710-64fb-48dd-abf8-064c85527502 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:28.836674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdf710-64fb-48dd-abf8-064c85527502 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:28.836951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdf710-64fb-48dd-abf8-064c85527502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:28.858095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a3bdf710-64fb-48dd-abf8-064c85527502 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:28.859323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a3bdf710-64fb-48dd-abf8-064c85527502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:28.867323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1300/2582] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:28.871550 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac40da-e106-40a7-89d7-7f62f3088deb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:28.881315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac40da-e106-40a7-89d7-7f62f3088deb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:28.887255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a7533-8666-4031-ad4b-7ccb7505d732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:28.887860 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a7533-8666-4031-ad4b-7ccb7505d732 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:28.888181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a7533-8666-4031-ad4b-7ccb7505d732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:28.888457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a7533-8666-4031-ad4b-7ccb7505d732 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:28.889115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a7533-8666-4031-ad4b-7ccb7505d732 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:38:28.889422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a7533-8666-4031-ad4b-7ccb7505d732 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:28.891014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a7533-8666-4031-ad4b-7ccb7505d732 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:28.904667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1283/2583] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:38:28.909264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdf710-64fb-48dd-abf8-064c85527502 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:28.918421 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdf710-64fb-48dd-abf8-064c85527502 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:28.923109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089053-4dc0-400d-a92e-a5bb50b0fdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:28.923720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089053-4dc0-400d-a92e-a5bb50b0fdf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:28.924001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089053-4dc0-400d-a92e-a5bb50b0fdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:28.924302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089053-4dc0-400d-a92e-a5bb50b0fdf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:28.924931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43089053-4dc0-400d-a92e-a5bb50b0fdf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:28.926858 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43089053-4dc0-400d-a92e-a5bb50b0fdf3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:28.979346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a7533-8666-4031-ad4b-7ccb7505d732 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:28.981758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1301/2584] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:38:28 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:28.982446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-996a7533-8666-4031-ad4b-7ccb7505d732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:28.985488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089053-4dc0-400d-a92e-a5bb50b0fdf3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:28.993649 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:29.056036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:29.062619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089053-4dc0-400d-a92e-a5bb50b0fdf3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:29.069486 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f7d1f-4596-47d7-bda7-93bf0ba162b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:29.070409 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7d1f-4596-47d7-bda7-93bf0ba162b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:29.071282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7d1f-4596-47d7-bda7-93bf0ba162b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:29.072568 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7d1f-4596-47d7-bda7-93bf0ba162b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:29.073650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f7d1f-4596-47d7-bda7-93bf0ba162b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:29.074200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a7533-8666-4031-ad4b-7ccb7505d732 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:29.076854 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f7d1f-4596-47d7-bda7-93bf0ba162b6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:29.102004 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1302/2585] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:38:28 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:38:29.106383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f7d1f-4596-47d7-bda7-93bf0ba162b6 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:29.118149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f7d1f-4596-47d7-bda7-93bf0ba162b6 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:29.123809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea149-2bed-4703-b2ba-2381143925d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:29.124787 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea149-2bed-4703-b2ba-2381143925d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:29.125511 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea149-2bed-4703-b2ba-2381143925d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:29.126091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea149-2bed-4703-b2ba-2381143925d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:29.127803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ea149-2bed-4703-b2ba-2381143925d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:29.130282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ea149-2bed-4703-b2ba-2381143925d3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:29.142757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1303/2586] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:38:29.146475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea149-2bed-4703-b2ba-2381143925d3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:29.151513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b47858-ef7f-4016-ae1e-2e0b3f7507f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:29.152164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b47858-ef7f-4016-ae1e-2e0b3f7507f7 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:29.152445 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b47858-ef7f-4016-ae1e-2e0b3f7507f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:29.152925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b47858-ef7f-4016-ae1e-2e0b3f7507f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:29.154951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f7b47858-ef7f-4016-ae1e-2e0b3f7507f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:29.159381 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea149-2bed-4703-b2ba-2381143925d3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:29.164835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d91a9-401d-4d8a-9699-dfa114f60495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:29.166212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d91a9-401d-4d8a-9699-dfa114f60495 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:29.166494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d91a9-401d-4d8a-9699-dfa114f60495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:29.166881 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d91a9-401d-4d8a-9699-dfa114f60495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:29.167567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d91a9-401d-4d8a-9699-dfa114f60495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:29.167868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b47858-ef7f-4016-ae1e-2e0b3f7507f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:29.169346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d91a9-401d-4d8a-9699-dfa114f60495 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:29.169660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b47858-ef7f-4016-ae1e-2e0b3f7507f7 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:29.181830 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1284/2587] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:29.184095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1304/2588] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:38:29.187034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b47858-ef7f-4016-ae1e-2e0b3f7507f7 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:29.198594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b47858-ef7f-4016-ae1e-2e0b3f7507f7 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:29.203826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948bccf-3575-4155-a2e4-296a9536836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:29.204869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948bccf-3575-4155-a2e4-296a9536836a admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:29.205415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948bccf-3575-4155-a2e4-296a9536836a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:29.205692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948bccf-3575-4155-a2e4-296a9536836a admin admin] PATH_INFO: `/v3/projects/8551aa3117164688b3c69f6c4a3a436d/users/950f7d2a46224bc0becdb6be4fce2dc5/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:29.222217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948bccf-3575-4155-a2e4-296a9536836a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8551aa3117164688b3c69f6c4a3a436d, user_id=950f7d2a46224bc0becdb6be4fce2dc5, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:29.225624 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948bccf-3575-4155-a2e4-296a9536836a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:29.239610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1285/2589] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:38:29 2022] PUT /identity/v3/projects/8551aa3117164688b3c69f6c4a3a436d/users/950f7d2a46224bc0becdb6be4fce2dc5/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:38:29.251695 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd0bfd-0c0a-4c9f-bfc5-34002552d175 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:29.251981 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd0bfd-0c0a-4c9f-bfc5-34002552d175 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:29.252309 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd0bfd-0c0a-4c9f-bfc5-34002552d175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:29.283232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-b3bd0bfd-0c0a-4c9f-bfc5-34002552d175 None None] MFA Rules not processed for user `950f7d2a46224bc0becdb6be4fce2dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:29.284685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b3bd0bfd-0c0a-4c9f-bfc5-34002552d175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:29.361128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1305/2590] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:29.372719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948bccf-3575-4155-a2e4-296a9536836a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:29.374483 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5948bccf-3575-4155-a2e4-296a9536836a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:29.437214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948bccf-3575-4155-a2e4-296a9536836a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:29.440884 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f149eb-0a4c-419c-880e-8c2d913b4700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:29.441546 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f149eb-0a4c-419c-880e-8c2d913b4700 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:29.441754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f149eb-0a4c-419c-880e-8c2d913b4700 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:29.442026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f149eb-0a4c-419c-880e-8c2d913b4700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:29.452030 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f149eb-0a4c-419c-880e-8c2d913b4700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:29.453499 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f149eb-0a4c-419c-880e-8c2d913b4700 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:29.464170 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1286/2591] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:37.917499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd0bfd-0c0a-4c9f-bfc5-34002552d175 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:37.985746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd0bfd-0c0a-4c9f-bfc5-34002552d175 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:37.990246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:37.990876 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:37.991160 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:37.991438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] PATH_INFO: `/v3/users/950f7d2a46224bc0becdb6be4fce2dc5` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:37.992608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] RBAC: Authorizing `identity:delete_user(user_id=950f7d2a46224bc0becdb6be4fce2dc5)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:37.994540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:38.041171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '950f7d2a46224bc0becdb6be4fce2dc5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:38.041826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '950f7d2a46224bc0becdb6be4fce2dc5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:38.042461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '950f7d2a46224bc0becdb6be4fce2dc5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:38.049715 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '950f7d2a46224bc0becdb6be4fce2dc5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:38.050928 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:38.059458 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '950f7d2a46224bc0becdb6be4fce2dc5'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:38.077454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1306/2592] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:38:37 2022] DELETE /identity/v3/users/950f7d2a46224bc0becdb6be4fce2dc5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:38.081971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e46e29-a68f-43e7-8f12-57123e8f7a12 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:38.082355 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e46e29-a68f-43e7-8f12-57123e8f7a12 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:38.082542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e46e29-a68f-43e7-8f12-57123e8f7a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:38.091227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:38.102781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-13e46e29-a68f-43e7-8f12-57123e8f7a12 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:38.103975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-13e46e29-a68f-43e7-8f12-57123e8f7a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:38.166272 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1287/2593] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:38.175918 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:38.177404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:38.238041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6deb5-a91c-4c02-bd90-9eb2fe2d442d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:38.242051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6060414-3106-4935-b9ac-a6cd1e0ccc12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:38.242647 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6060414-3106-4935-b9ac-a6cd1e0ccc12 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:38.242923 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6060414-3106-4935-b9ac-a6cd1e0ccc12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:38.243195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6060414-3106-4935-b9ac-a6cd1e0ccc12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:38.253112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6060414-3106-4935-b9ac-a6cd1e0ccc12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:38.254556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6060414-3106-4935-b9ac-a6cd1e0ccc12 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:38.265008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1307/2594] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:38.533979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e46e29-a68f-43e7-8f12-57123e8f7a12 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:38.535486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-13e46e29-a68f-43e7-8f12-57123e8f7a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:38.554834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e46e29-a68f-43e7-8f12-57123e8f7a12 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:38.561027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:38.561996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:38.562305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:38.562719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] PATH_INFO: `/v3/projects/8551aa3117164688b3c69f6c4a3a436d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:38.563938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8551aa3117164688b3c69f6c4a3a436d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:38.565906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:38.617341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8551aa3117164688b3c69f6c4a3a436d', 'request_id': 'req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:38.624363 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8551aa3117164688b3c69f6c4a3a436d', 'request_id': 'req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:38.630183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8551aa3117164688b3c69f6c4a3a436d', 'request_id': 'req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:38.643151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] The token cache is being invalidate because project 8551aa3117164688b3c69f6c4a3a436d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:38:38.643443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:38.644129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ad164cb3-248f-45a7-8ed0-ea4fb73a9029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:38.645609 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1288/2595] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:38:38 2022] DELETE /identity/v3/projects/8551aa3117164688b3c69f6c4a3a436d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:38.650465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95362-be67-4b82-825e-9350c049c55a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:38.650978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95362-be67-4b82-825e-9350c049c55a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:38.651258 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95362-be67-4b82-825e-9350c049c55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:38.672277 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6de95362-be67-4b82-825e-9350c049c55a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:38.673504 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6de95362-be67-4b82-825e-9350c049c55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:38.737691 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1308/2596] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:38.745669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d217df-6d2d-4ff1-a1b1-066d3785a212 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:38.746474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d217df-6d2d-4ff1-a1b1-066d3785a212 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:38.746878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d217df-6d2d-4ff1-a1b1-066d3785a212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:38.767080 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-89d217df-6d2d-4ff1-a1b1-066d3785a212 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:38.768390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-89d217df-6d2d-4ff1-a1b1-066d3785a212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:38.780798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1289/2597] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:38.785535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de95362-be67-4b82-825e-9350c049c55a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:38.796128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de95362-be67-4b82-825e-9350c049c55a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:38.800752 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51fb24-ad06-4b94-878f-16eef63679eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:38.801340 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51fb24-ad06-4b94-878f-16eef63679eb admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:38.801619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51fb24-ad06-4b94-878f-16eef63679eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:38.801886 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51fb24-ad06-4b94-878f-16eef63679eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:38.802552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51fb24-ad06-4b94-878f-16eef63679eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:38:38.802853 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51fb24-ad06-4b94-878f-16eef63679eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:38.804493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51fb24-ad06-4b94-878f-16eef63679eb admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:38.816432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1309/2598] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:38:38.820793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d217df-6d2d-4ff1-a1b1-066d3785a212 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:38.830234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d217df-6d2d-4ff1-a1b1-066d3785a212 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:38.835065 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad7c94-df3a-465f-9432-44a4d75fccb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:38.835658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad7c94-df3a-465f-9432-44a4d75fccb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:38.835941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad7c94-df3a-465f-9432-44a4d75fccb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:38.836252 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad7c94-df3a-465f-9432-44a4d75fccb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:38.836880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad7c94-df3a-465f-9432-44a4d75fccb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:38.838793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad7c94-df3a-465f-9432-44a4d75fccb9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:38.866205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1290/2599] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:38:38 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:38:38.870533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51fb24-ad06-4b94-878f-16eef63679eb admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:38.881373 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51fb24-ad06-4b94-878f-16eef63679eb admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:38.938677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877b33f-d465-4db9-9008-e9f589b08b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:38.939336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877b33f-d465-4db9-9008-e9f589b08b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:38.939570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877b33f-d465-4db9-9008-e9f589b08b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:38.939839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877b33f-d465-4db9-9008-e9f589b08b53 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:38.940490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877b33f-d465-4db9-9008-e9f589b08b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:38.942401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877b33f-d465-4db9-9008-e9f589b08b53 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:38.951212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:38.965260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1310/2600] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:38:38 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:38.969290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad7c94-df3a-465f-9432-44a4d75fccb9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:38.981187 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad7c94-df3a-465f-9432-44a4d75fccb9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:38.998982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcb48c-d02b-40fa-af83-8aa2397788ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:38.999702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcb48c-d02b-40fa-af83-8aa2397788ce admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:38.999992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcb48c-d02b-40fa-af83-8aa2397788ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.000291 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcb48c-d02b-40fa-af83-8aa2397788ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.001003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcb48c-d02b-40fa-af83-8aa2397788ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:39.002895 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcb48c-d02b-40fa-af83-8aa2397788ce admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:39.022236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1291/2601] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:38:39.023649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877b33f-d465-4db9-9008-e9f589b08b53 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:39.034303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877b33f-d465-4db9-9008-e9f589b08b53 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:39.039109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8864e-c1eb-4fa0-91e2-6a67497c48c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:39.039729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8864e-c1eb-4fa0-91e2-6a67497c48c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:39.040023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8864e-c1eb-4fa0-91e2-6a67497c48c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.040335 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8864e-c1eb-4fa0-91e2-6a67497c48c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.040989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8864e-c1eb-4fa0-91e2-6a67497c48c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:39.042609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8864e-c1eb-4fa0-91e2-6a67497c48c8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:39.053784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1311/2602] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:38:39.058801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcb48c-d02b-40fa-af83-8aa2397788ce admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:39.069542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcb48c-d02b-40fa-af83-8aa2397788ce admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:39.074814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944303-5e18-4302-9320-85449305460d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:39.075467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944303-5e18-4302-9320-85449305460d admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:39.075755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944303-5e18-4302-9320-85449305460d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.076029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944303-5e18-4302-9320-85449305460d admin admin] PATH_INFO: `/v3/projects/0803cb4e31ed43e8b07ccd628804ce9f/users/0bc8b4528da74ee9bb436b01de76c145/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.092882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7944303-5e18-4302-9320-85449305460d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0803cb4e31ed43e8b07ccd628804ce9f, user_id=0bc8b4528da74ee9bb436b01de76c145, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:39.095567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7944303-5e18-4302-9320-85449305460d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:39.111562 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1292/2603] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:38:39 2022] PUT /identity/v3/projects/0803cb4e31ed43e8b07ccd628804ce9f/users/0bc8b4528da74ee9bb436b01de76c145/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:39.122739 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b944ba-0776-4e66-b07e-6cbc2087c24c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:39.123030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b944ba-0776-4e66-b07e-6cbc2087c24c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.123308 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b944ba-0776-4e66-b07e-6cbc2087c24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.154643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-86b944ba-0776-4e66-b07e-6cbc2087c24c None None] MFA Rules not processed for user `0bc8b4528da74ee9bb436b01de76c145`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:39.155882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-86b944ba-0776-4e66-b07e-6cbc2087c24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:39.230511 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1312/2604] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:39.235063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944303-5e18-4302-9320-85449305460d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:39.263726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:39.309823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944303-5e18-4302-9320-85449305460d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:39.314483 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac11da-4655-47ea-bf3e-99de82e6fbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:39.315091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac11da-4655-47ea-bf3e-99de82e6fbbc admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:39.315368 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac11da-4655-47ea-bf3e-99de82e6fbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.315638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac11da-4655-47ea-bf3e-99de82e6fbbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.316301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac11da-4655-47ea-bf3e-99de82e6fbbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:39.318210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac11da-4655-47ea-bf3e-99de82e6fbbc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:39.352819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1293/2605] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:38:39 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:39.357503 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b944ba-0776-4e66-b07e-6cbc2087c24c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:39.368320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b944ba-0776-4e66-b07e-6cbc2087c24c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:39.425663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66282df-94f9-4567-a0ee-0a4dd94c38d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:39.426260 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66282df-94f9-4567-a0ee-0a4dd94c38d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:39.426543 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66282df-94f9-4567-a0ee-0a4dd94c38d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.426818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66282df-94f9-4567-a0ee-0a4dd94c38d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.427453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66282df-94f9-4567-a0ee-0a4dd94c38d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:39.429425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66282df-94f9-4567-a0ee-0a4dd94c38d9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:39.450497 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1313/2606] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:38:39 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:39.454692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac11da-4655-47ea-bf3e-99de82e6fbbc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:39.465016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac11da-4655-47ea-bf3e-99de82e6fbbc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:39.469554 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037036d-e23f-4f6b-931a-5ff7bdd46c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:39.470138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037036d-e23f-4f6b-931a-5ff7bdd46c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:39.470414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037036d-e23f-4f6b-931a-5ff7bdd46c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.470685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037036d-e23f-4f6b-931a-5ff7bdd46c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.471316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037036d-e23f-4f6b-931a-5ff7bdd46c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:39.473058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037036d-e23f-4f6b-931a-5ff7bdd46c8a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:39.483893 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1294/2607] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:38:39.488366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66282df-94f9-4567-a0ee-0a4dd94c38d9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:39.499238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66282df-94f9-4567-a0ee-0a4dd94c38d9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:39.503889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88408375-d4f2-412d-8748-3d0e60fe797d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:39.504563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88408375-d4f2-412d-8748-3d0e60fe797d admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:39.504847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88408375-d4f2-412d-8748-3d0e60fe797d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.505125 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88408375-d4f2-412d-8748-3d0e60fe797d admin admin] PATH_INFO: `/v3/projects/537126a711ac45549e3944f1dd1530c1/users/e2f9af046dbc4354a4bd517f31358968/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.520320 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88408375-d4f2-412d-8748-3d0e60fe797d admin admin] RBAC: Authorizing `identity:create_grant(project_id=537126a711ac45549e3944f1dd1530c1, user_id=e2f9af046dbc4354a4bd517f31358968, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:39.522968 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88408375-d4f2-412d-8748-3d0e60fe797d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:39.535948 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1314/2608] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:38:39 2022] PUT /identity/v3/projects/537126a711ac45549e3944f1dd1530c1/users/e2f9af046dbc4354a4bd517f31358968/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:39.539889 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037036d-e23f-4f6b-931a-5ff7bdd46c8a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:39.550328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037036d-e23f-4f6b-931a-5ff7bdd46c8a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:39.607484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1993204-b266-4d18-8606-186a3fa5d6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:39.608117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1993204-b266-4d18-8606-186a3fa5d6dc admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:39.608402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1993204-b266-4d18-8606-186a3fa5d6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.608677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1993204-b266-4d18-8606-186a3fa5d6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.609314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1993204-b266-4d18-8606-186a3fa5d6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:39.610913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1993204-b266-4d18-8606-186a3fa5d6dc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:39.621427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1295/2609] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:38:39.625466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88408375-d4f2-412d-8748-3d0e60fe797d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:39.635612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88408375-d4f2-412d-8748-3d0e60fe797d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:39.640222 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc2572-a48a-426b-8687-f55568ae7bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:39.640855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc2572-a48a-426b-8687-f55568ae7bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:39.641130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc2572-a48a-426b-8687-f55568ae7bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.641400 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc2572-a48a-426b-8687-f55568ae7bd3 admin admin] PATH_INFO: `/v3/projects/537126a711ac45549e3944f1dd1530c1/users/e2f9af046dbc4354a4bd517f31358968/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.643272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc2572-a48a-426b-8687-f55568ae7bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=537126a711ac45549e3944f1dd1530c1, user_id=e2f9af046dbc4354a4bd517f31358968, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:39.645941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc2572-a48a-426b-8687-f55568ae7bd3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:39.656403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1315/2610] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:38:39 2022] PUT /identity/v3/projects/537126a711ac45549e3944f1dd1530c1/users/e2f9af046dbc4354a4bd517f31358968/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:39.668432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73cabb-5725-4b9c-a599-a0881dbc19e3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:39.668725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73cabb-5725-4b9c-a599-a0881dbc19e3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.669037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73cabb-5725-4b9c-a599-a0881dbc19e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.697739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5e73cabb-5725-4b9c-a599-a0881dbc19e3 None None] MFA Rules not processed for user `e2f9af046dbc4354a4bd517f31358968`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:39.698953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5e73cabb-5725-4b9c-a599-a0881dbc19e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:39.766693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1296/2611] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:39.778998 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc2572-a48a-426b-8687-f55568ae7bd3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:39.781076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1ecc2572-a48a-426b-8687-f55568ae7bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:39.808589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:39.842404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc2572-a48a-426b-8687-f55568ae7bd3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:39.846155 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e463b50-293f-4dc1-86c6-01c46723880e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:39.846761 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e463b50-293f-4dc1-86c6-01c46723880e service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:39.847041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e463b50-293f-4dc1-86c6-01c46723880e service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:39.847314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e463b50-293f-4dc1-86c6-01c46723880e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:39.857038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e463b50-293f-4dc1-86c6-01c46723880e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:39.858484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e463b50-293f-4dc1-86c6-01c46723880e service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:39.867946 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1316/2612] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:40.278776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73cabb-5725-4b9c-a599-a0881dbc19e3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:40.300186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:40.342075 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73cabb-5725-4b9c-a599-a0881dbc19e3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:40.346643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:40.347265 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:40.347548 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:40.347823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] PATH_INFO: `/v3/users/0bc8b4528da74ee9bb436b01de76c145` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:40.348982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bc8b4528da74ee9bb436b01de76c145)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:40.350908 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:40.398374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bc8b4528da74ee9bb436b01de76c145'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:40.399019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bc8b4528da74ee9bb436b01de76c145'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:40.399638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bc8b4528da74ee9bb436b01de76c145'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:40.407458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bc8b4528da74ee9bb436b01de76c145'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:40.414263 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bc8b4528da74ee9bb436b01de76c145'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:40.432070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1297/2613] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:38:40 2022] DELETE /identity/v3/users/0bc8b4528da74ee9bb436b01de76c145 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:40.436594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b644219-89ad-46be-a359-82db77198aa3 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:40.437095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b644219-89ad-46be-a359-82db77198aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:40.437203 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b644219-89ad-46be-a359-82db77198aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:40.455636 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5b644219-89ad-46be-a359-82db77198aa3 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:40.456892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5b644219-89ad-46be-a359-82db77198aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:40.516593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1317/2614] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:40.526328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:40.528256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:40.584380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2986e5-0482-4cf4-b319-308271d015ef admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:40.588168 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae37f7-be34-4565-8061-69a1cd87b199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:40.588767 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae37f7-be34-4565-8061-69a1cd87b199 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:40.589042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae37f7-be34-4565-8061-69a1cd87b199 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:40.589312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae37f7-be34-4565-8061-69a1cd87b199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:40.598897 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae37f7-be34-4565-8061-69a1cd87b199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:40.600358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae37f7-be34-4565-8061-69a1cd87b199 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:40.610191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1298/2615] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:40.887152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b644219-89ad-46be-a359-82db77198aa3 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:40.888703 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5b644219-89ad-46be-a359-82db77198aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:40.903399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b644219-89ad-46be-a359-82db77198aa3 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:40.908465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:40.909118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:40.909398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:40.909671 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] PATH_INFO: `/v3/projects/0803cb4e31ed43e8b07ccd628804ce9f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:40.910895 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0803cb4e31ed43e8b07ccd628804ce9f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:40.913170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:40.962077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0803cb4e31ed43e8b07ccd628804ce9f', 'request_id': 'req-8bf2a541-3020-4e75-bd1b-c193f120ece2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:40.969071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0803cb4e31ed43e8b07ccd628804ce9f', 'request_id': 'req-8bf2a541-3020-4e75-bd1b-c193f120ece2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:40.974598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0803cb4e31ed43e8b07ccd628804ce9f', 'request_id': 'req-8bf2a541-3020-4e75-bd1b-c193f120ece2'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:40.986886 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] The token cache is being invalidate because project 0803cb4e31ed43e8b07ccd628804ce9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:38:40.987170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:40.987777 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:40.989270 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1318/2616] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:38:40 2022] DELETE /identity/v3/projects/0803cb4e31ed43e8b07ccd628804ce9f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:40.993365 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae37f7-be34-4565-8061-69a1cd87b199 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:41.002812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae37f7-be34-4565-8061-69a1cd87b199 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:41.042650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:41.060395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55c695-d455-453c-840a-c78147164d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:41.061013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55c695-d455-453c-840a-c78147164d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:41.061290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55c695-d455-453c-840a-c78147164d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:41.061610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55c695-d455-453c-840a-c78147164d50 admin admin] PATH_INFO: `/v3/users/e2f9af046dbc4354a4bd517f31358968` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:41.062658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55c695-d455-453c-840a-c78147164d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2f9af046dbc4354a4bd517f31358968)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:41.064596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55c695-d455-453c-840a-c78147164d50 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:41.109468 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-af55c695-d455-453c-840a-c78147164d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2f9af046dbc4354a4bd517f31358968'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:41.110123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-af55c695-d455-453c-840a-c78147164d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2f9af046dbc4354a4bd517f31358968'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:41.110729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-af55c695-d455-453c-840a-c78147164d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2f9af046dbc4354a4bd517f31358968'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:41.118924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-af55c695-d455-453c-840a-c78147164d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2f9af046dbc4354a4bd517f31358968'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:41.126387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-af55c695-d455-453c-840a-c78147164d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2f9af046dbc4354a4bd517f31358968'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:41.144069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1299/2617] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:38:40 2022] DELETE /identity/v3/users/e2f9af046dbc4354a4bd517f31358968 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:38:41.420072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:41.425213 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:41.434821 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2a541-3020-4e75-bd1b-c193f120ece2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:41.481758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:41.495999 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:41.496685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:41.496966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:41.497238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] PATH_INFO: `/v3/projects/537126a711ac45549e3944f1dd1530c1` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:41.498467 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=537126a711ac45549e3944f1dd1530c1)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:41.500475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:41.546327 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '537126a711ac45549e3944f1dd1530c1', 'request_id': 'req-b60a114b-faf2-4370-ba4a-a1e0a07207b9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:41.552299 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '537126a711ac45549e3944f1dd1530c1', 'request_id': 'req-b60a114b-faf2-4370-ba4a-a1e0a07207b9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:41.557654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '537126a711ac45549e3944f1dd1530c1', 'request_id': 'req-b60a114b-faf2-4370-ba4a-a1e0a07207b9'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:41.570076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] The token cache is being invalidate because project 537126a711ac45549e3944f1dd1530c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:38:41.570350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:41.570980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b60a114b-faf2-4370-ba4a-a1e0a07207b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:41.572526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1319/2618] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:38:41 2022] DELETE /identity/v3/projects/537126a711ac45549e3944f1dd1530c1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:38:41.577804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7cfff-530b-4a5c-9692-275ba9aa23c7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:41.578293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7cfff-530b-4a5c-9692-275ba9aa23c7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:41.578427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7cfff-530b-4a5c-9692-275ba9aa23c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:41.598870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3ef7cfff-530b-4a5c-9692-275ba9aa23c7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:41.600096 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3ef7cfff-530b-4a5c-9692-275ba9aa23c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:41.660882 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1300/2619] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:41.668583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a2114-4fea-4382-b6c7-d51e60549a30 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:41.668870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a2114-4fea-4382-b6c7-d51e60549a30 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:41.669142 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a2114-4fea-4382-b6c7-d51e60549a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:41.696738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-fe7a2114-4fea-4382-b6c7-d51e60549a30 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:41.697952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fe7a2114-4fea-4382-b6c7-d51e60549a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:41.706533 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1320/2620] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:41.719628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7cfff-530b-4a5c-9692-275ba9aa23c7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:41.729369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7cfff-530b-4a5c-9692-275ba9aa23c7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:41.733877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921612ff-5c65-4535-84e4-c7feb2c9e0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:41.734463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921612ff-5c65-4535-84e4-c7feb2c9e0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:41.734737 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921612ff-5c65-4535-84e4-c7feb2c9e0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:41.735003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921612ff-5c65-4535-84e4-c7feb2c9e0d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:41.735640 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921612ff-5c65-4535-84e4-c7feb2c9e0d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:38:41.735940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921612ff-5c65-4535-84e4-c7feb2c9e0d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:41.737581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921612ff-5c65-4535-84e4-c7feb2c9e0d2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:41.748969 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1301/2621] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:38:41.753440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a2114-4fea-4382-b6c7-d51e60549a30 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:41.763343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a2114-4fea-4382-b6c7-d51e60549a30 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:41.768021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f93559-e342-4604-bff7-fc78b82493ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:41.768656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f93559-e342-4604-bff7-fc78b82493ce admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:41.768937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f93559-e342-4604-bff7-fc78b82493ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:41.769210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f93559-e342-4604-bff7-fc78b82493ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:41.769878 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f93559-e342-4604-bff7-fc78b82493ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:41.771787 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f93559-e342-4604-bff7-fc78b82493ce admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:41.804397 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1321/2622] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:38:41 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:41.806374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921612ff-5c65-4535-84e4-c7feb2c9e0d2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:41.816508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921612ff-5c65-4535-84e4-c7feb2c9e0d2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:41.860474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:41.874864 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb1fc8-ed03-4ab0-b7eb-9e730b016c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:41.875473 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1fc8-ed03-4ab0-b7eb-9e730b016c31 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:41.875757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1fc8-ed03-4ab0-b7eb-9e730b016c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:41.876029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1fc8-ed03-4ab0-b7eb-9e730b016c31 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:41.876700 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb1fc8-ed03-4ab0-b7eb-9e730b016c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:41.878615 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb1fc8-ed03-4ab0-b7eb-9e730b016c31 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:41.904476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1302/2623] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:38:41 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:41.905899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f93559-e342-4604-bff7-fc78b82493ce admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:41.917823 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f93559-e342-4604-bff7-fc78b82493ce admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:41.922378 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59902ec4-2928-47e5-a848-60e6a62518b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:41.922970 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59902ec4-2928-47e5-a848-60e6a62518b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:41.923265 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59902ec4-2928-47e5-a848-60e6a62518b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:41.923511 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59902ec4-2928-47e5-a848-60e6a62518b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:41.924202 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59902ec4-2928-47e5-a848-60e6a62518b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:41.925788 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59902ec4-2928-47e5-a848-60e6a62518b8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:41.936027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1322/2624] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:38:41.939983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb1fc8-ed03-4ab0-b7eb-9e730b016c31 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:41.950781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb1fc8-ed03-4ab0-b7eb-9e730b016c31 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:41.955429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ee10d-d29b-4083-8ab6-92abe2f8703f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:41.956027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ee10d-d29b-4083-8ab6-92abe2f8703f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:41.956345 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ee10d-d29b-4083-8ab6-92abe2f8703f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:41.956646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ee10d-d29b-4083-8ab6-92abe2f8703f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:41.957290 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ee10d-d29b-4083-8ab6-92abe2f8703f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:41.958883 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ee10d-d29b-4083-8ab6-92abe2f8703f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:41.969795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1303/2625] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:38:41.974117 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59902ec4-2928-47e5-a848-60e6a62518b8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:41.983989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59902ec4-2928-47e5-a848-60e6a62518b8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:41.988844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecb581-1141-4365-a5cd-1f2a01c8553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:41.989522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecb581-1141-4365-a5cd-1f2a01c8553e admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:41.989803 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecb581-1141-4365-a5cd-1f2a01c8553e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:41.990075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecb581-1141-4365-a5cd-1f2a01c8553e admin admin] PATH_INFO: `/v3/projects/84ec545c326d4892a5a2c2d6cbbb27fa/users/58cca120feb84767ae037824364c2ff7/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:42.005663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecb581-1141-4365-a5cd-1f2a01c8553e admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ec545c326d4892a5a2c2d6cbbb27fa, user_id=58cca120feb84767ae037824364c2ff7, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:42.008350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecb581-1141-4365-a5cd-1f2a01c8553e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:42.022205 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1323/2626] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:38:41 2022] PUT /identity/v3/projects/84ec545c326d4892a5a2c2d6cbbb27fa/users/58cca120feb84767ae037824364c2ff7/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:42.032731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e126-6fd5-4cae-aabf-6aae70589915 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:42.033015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e126-6fd5-4cae-aabf-6aae70589915 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:42.033315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e126-6fd5-4cae-aabf-6aae70589915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:42.060692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ddf5e126-6fd5-4cae-aabf-6aae70589915 None None] MFA Rules not processed for user `58cca120feb84767ae037824364c2ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:42.061896 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ddf5e126-6fd5-4cae-aabf-6aae70589915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:42.124014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1304/2627] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:42.128448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecb581-1141-4365-a5cd-1f2a01c8553e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:42.188773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecb581-1141-4365-a5cd-1f2a01c8553e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:42.193363 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aaf08-e1aa-45cf-a17e-2f20c1523d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:42.193952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aaf08-e1aa-45cf-a17e-2f20c1523d05 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:42.194232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aaf08-e1aa-45cf-a17e-2f20c1523d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:42.194508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aaf08-e1aa-45cf-a17e-2f20c1523d05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:42.195127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4aaf08-e1aa-45cf-a17e-2f20c1523d05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:42.197067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4aaf08-e1aa-45cf-a17e-2f20c1523d05 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:42.217710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:42.224756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1324/2628] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:38:42 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:42.228942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5e126-6fd5-4cae-aabf-6aae70589915 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:42.240018 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5e126-6fd5-4cae-aabf-6aae70589915 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:42.297755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429993c9-f276-4abf-80cd-eec96b6075fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:42.298456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429993c9-f276-4abf-80cd-eec96b6075fb admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:42.298634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429993c9-f276-4abf-80cd-eec96b6075fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:42.298943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429993c9-f276-4abf-80cd-eec96b6075fb admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:42.299581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429993c9-f276-4abf-80cd-eec96b6075fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:42.301686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429993c9-f276-4abf-80cd-eec96b6075fb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:42.322881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1305/2629] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:38:42 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:42.326780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aaf08-e1aa-45cf-a17e-2f20c1523d05 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:42.337151 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aaf08-e1aa-45cf-a17e-2f20c1523d05 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:42.341762 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668e845-7c43-4d82-b54b-740b7f95fa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:42.342352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668e845-7c43-4d82-b54b-740b7f95fa86 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:42.342635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668e845-7c43-4d82-b54b-740b7f95fa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:42.342913 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668e845-7c43-4d82-b54b-740b7f95fa86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:42.343554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668e845-7c43-4d82-b54b-740b7f95fa86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:42.345195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668e845-7c43-4d82-b54b-740b7f95fa86 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:42.356239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1325/2630] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:38:42.360247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429993c9-f276-4abf-80cd-eec96b6075fb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:42.370606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429993c9-f276-4abf-80cd-eec96b6075fb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:42.376875 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b5237-27fc-462c-9e1b-e71e84a4fdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:42.377520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b5237-27fc-462c-9e1b-e71e84a4fdab admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:42.377811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b5237-27fc-462c-9e1b-e71e84a4fdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:42.378090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b5237-27fc-462c-9e1b-e71e84a4fdab admin admin] PATH_INFO: `/v3/projects/d7898087bfc247d2bb859990d03a3a1e/users/2ef64448156c408dbdfe4fa269e680d2/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:42.393658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b5237-27fc-462c-9e1b-e71e84a4fdab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7898087bfc247d2bb859990d03a3a1e, user_id=2ef64448156c408dbdfe4fa269e680d2, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:42.396354 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b5237-27fc-462c-9e1b-e71e84a4fdab admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:42.409943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1306/2631] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:38:42 2022] PUT /identity/v3/projects/d7898087bfc247d2bb859990d03a3a1e/users/2ef64448156c408dbdfe4fa269e680d2/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:42.413869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668e845-7c43-4d82-b54b-740b7f95fa86 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:42.424377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668e845-7c43-4d82-b54b-740b7f95fa86 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:42.479774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb8405-a983-4b83-81cb-00475d681e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:42.481007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8405-a983-4b83-81cb-00475d681e24 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:42.481338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8405-a983-4b83-81cb-00475d681e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:42.481639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8405-a983-4b83-81cb-00475d681e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:42.482483 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb8405-a983-4b83-81cb-00475d681e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:42.484710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb8405-a983-4b83-81cb-00475d681e24 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:42.495769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1326/2632] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:38:42.500643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b5237-27fc-462c-9e1b-e71e84a4fdab admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:42.510798 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b5237-27fc-462c-9e1b-e71e84a4fdab admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:42.515359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d11cd-808c-4cff-9da3-fbcb226fb3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:42.515991 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d11cd-808c-4cff-9da3-fbcb226fb3ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:42.516313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d11cd-808c-4cff-9da3-fbcb226fb3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:42.516568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d11cd-808c-4cff-9da3-fbcb226fb3ee admin admin] PATH_INFO: `/v3/projects/d7898087bfc247d2bb859990d03a3a1e/users/2ef64448156c408dbdfe4fa269e680d2/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:42.518453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d11cd-808c-4cff-9da3-fbcb226fb3ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7898087bfc247d2bb859990d03a3a1e, user_id=2ef64448156c408dbdfe4fa269e680d2, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:42.521114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d11cd-808c-4cff-9da3-fbcb226fb3ee admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:42.531081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1307/2633] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:38:42 2022] PUT /identity/v3/projects/d7898087bfc247d2bb859990d03a3a1e/users/2ef64448156c408dbdfe4fa269e680d2/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:42.596036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f681b8f-2a0b-4feb-ac5e-7b9d9153e6f4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:42.596353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f681b8f-2a0b-4feb-ac5e-7b9d9153e6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:42.596632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f681b8f-2a0b-4feb-ac5e-7b9d9153e6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:42.625302 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2f681b8f-2a0b-4feb-ac5e-7b9d9153e6f4 None None] MFA Rules not processed for user `2ef64448156c408dbdfe4fa269e680d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:42.626522 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2f681b8f-2a0b-4feb-ac5e-7b9d9153e6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:42.693240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1327/2634] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:42.704351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d11cd-808c-4cff-9da3-fbcb226fb3ee admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:42.706059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-374d11cd-808c-4cff-9da3-fbcb226fb3ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:42.717486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:42.764728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d11cd-808c-4cff-9da3-fbcb226fb3ee admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:42.768990 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82480dcc-2827-4c2a-9374-66f252c1f1be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:42.769682 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82480dcc-2827-4c2a-9374-66f252c1f1be service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:42.769964 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82480dcc-2827-4c2a-9374-66f252c1f1be service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:42.770306 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82480dcc-2827-4c2a-9374-66f252c1f1be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:42.781302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82480dcc-2827-4c2a-9374-66f252c1f1be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:42.782994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82480dcc-2827-4c2a-9374-66f252c1f1be service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:42.793612 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1308/2635] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:42.867055 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f681b8f-2a0b-4feb-ac5e-7b9d9153e6f4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:42.876609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f681b8f-2a0b-4feb-ac5e-7b9d9153e6f4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:42.880172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef843f-90fd-4ed7-9b3d-5af0b55733c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:42.880758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef843f-90fd-4ed7-9b3d-5af0b55733c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:42.881033 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef843f-90fd-4ed7-9b3d-5af0b55733c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:42.881303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef843f-90fd-4ed7-9b3d-5af0b55733c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:42.937627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef843f-90fd-4ed7-9b3d-5af0b55733c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:42.939079 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef843f-90fd-4ed7-9b3d-5af0b55733c1 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:42.950373 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1328/2636] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:52.789442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=22 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:52.943918 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=96 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:55.309539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82480dcc-2827-4c2a-9374-66f252c1f1be service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:55.373247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82480dcc-2827-4c2a-9374-66f252c1f1be service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:55.377797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:55.378424 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:55.378705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:55.378982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] PATH_INFO: `/v3/users/58cca120feb84767ae037824364c2ff7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:55.380148 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58cca120feb84767ae037824364c2ff7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:55.382085 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:55.430491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58cca120feb84767ae037824364c2ff7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:55.431149 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58cca120feb84767ae037824364c2ff7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:55.431764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58cca120feb84767ae037824364c2ff7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:55.440423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58cca120feb84767ae037824364c2ff7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:55.447610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58cca120feb84767ae037824364c2ff7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:55.465480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1309/2637] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:38:55 2022] DELETE /identity/v3/users/58cca120feb84767ae037824364c2ff7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:55.469957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41368-9ba6-4d2a-a701-963b16bd9ed4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:55.470441 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41368-9ba6-4d2a-a701-963b16bd9ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:55.470632 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41368-9ba6-4d2a-a701-963b16bd9ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:55.492797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-def41368-9ba6-4d2a-a701-963b16bd9ed4 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:55.494193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-def41368-9ba6-4d2a-a701-963b16bd9ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:55.562901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1329/2638] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:55.569453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:55.571049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:55.644131 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2aa3c9-766e-4dac-88a3-3f82d8688969 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:55.647776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae75eda-0687-432e-9fdc-81dd5236cffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:55.648433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae75eda-0687-432e-9fdc-81dd5236cffb service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:55.648697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae75eda-0687-432e-9fdc-81dd5236cffb service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:55.648973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae75eda-0687-432e-9fdc-81dd5236cffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:55.658463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae75eda-0687-432e-9fdc-81dd5236cffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:55.659894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae75eda-0687-432e-9fdc-81dd5236cffb service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:55.670502 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1310/2639] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:55.991770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def41368-9ba6-4d2a-a701-963b16bd9ed4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:55.993555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-def41368-9ba6-4d2a-a701-963b16bd9ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:56.010311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def41368-9ba6-4d2a-a701-963b16bd9ed4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:56.015664 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:56.016474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:56.016905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:56.017176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] PATH_INFO: `/v3/projects/84ec545c326d4892a5a2c2d6cbbb27fa` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:56.018570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ec545c326d4892a5a2c2d6cbbb27fa)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:56.021095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:56.068317 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ec545c326d4892a5a2c2d6cbbb27fa', 'request_id': 'req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.074551 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ec545c326d4892a5a2c2d6cbbb27fa', 'request_id': 'req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.080555 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ec545c326d4892a5a2c2d6cbbb27fa', 'request_id': 'req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.093795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] The token cache is being invalidate because project 84ec545c326d4892a5a2c2d6cbbb27fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:38:56.094091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.094727 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.096236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1330/2640] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:38:55 2022] DELETE /identity/v3/projects/84ec545c326d4892a5a2c2d6cbbb27fa => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:56.100264 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae75eda-0687-432e-9fdc-81dd5236cffb service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:56.109744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae75eda-0687-432e-9fdc-81dd5236cffb service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:56.127818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:56.173091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ce2b0-c663-4674-9c44-93485100296f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:56.173723 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ce2b0-c663-4674-9c44-93485100296f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:56.174012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ce2b0-c663-4674-9c44-93485100296f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:56.174287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ce2b0-c663-4674-9c44-93485100296f admin admin] PATH_INFO: `/v3/users/2ef64448156c408dbdfe4fa269e680d2` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:56.175417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ce2b0-c663-4674-9c44-93485100296f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef64448156c408dbdfe4fa269e680d2)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:56.177395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ce2b0-c663-4674-9c44-93485100296f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:56.226436 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-124ce2b0-c663-4674-9c44-93485100296f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef64448156c408dbdfe4fa269e680d2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.227971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-124ce2b0-c663-4674-9c44-93485100296f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef64448156c408dbdfe4fa269e680d2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.228662 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-124ce2b0-c663-4674-9c44-93485100296f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef64448156c408dbdfe4fa269e680d2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.237834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-124ce2b0-c663-4674-9c44-93485100296f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef64448156c408dbdfe4fa269e680d2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.244993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-124ce2b0-c663-4674-9c44-93485100296f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef64448156c408dbdfe4fa269e680d2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.262892 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1311/2641] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:38:56 2022] DELETE /identity/v3/users/2ef64448156c408dbdfe4fa269e680d2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:56.541092 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:56.542702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:56.556985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10680a-1a8d-4494-8509-8e93f0e2bc3a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:56.615357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:56.615984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:56.616295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:56.616569 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] PATH_INFO: `/v3/projects/d7898087bfc247d2bb859990d03a3a1e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:56.617746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7898087bfc247d2bb859990d03a3a1e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:56.619671 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:56.655896 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:56.673844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7898087bfc247d2bb859990d03a3a1e', 'request_id': 'req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.679910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7898087bfc247d2bb859990d03a3a1e', 'request_id': 'req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.685435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7898087bfc247d2bb859990d03a3a1e', 'request_id': 'req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.697370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] The token cache is being invalidate because project d7898087bfc247d2bb859990d03a3a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:38:56.697657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.698266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e83dea58-ccf8-41b0-9aae-ae7f330b6dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:38:56.699726 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1331/2642] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:38:56 2022] DELETE /identity/v3/projects/d7898087bfc247d2bb859990d03a3a1e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:56.705034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4ef85-e839-489c-8af7-e08d4d7af9a2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:56.705542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4ef85-e839-489c-8af7-e08d4d7af9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:56.705666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4ef85-e839-489c-8af7-e08d4d7af9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:56.724804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d8e4ef85-e839-489c-8af7-e08d4d7af9a2 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:56.725996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d8e4ef85-e839-489c-8af7-e08d4d7af9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:56.783510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1312/2643] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:56.791032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ace29b-5226-4823-8f8b-c258491e7dab None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:56.791318 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ace29b-5226-4823-8f8b-c258491e7dab None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:56.791590 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ace29b-5226-4823-8f8b-c258491e7dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:56.810869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-47ace29b-5226-4823-8f8b-c258491e7dab None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:56.812100 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-47ace29b-5226-4823-8f8b-c258491e7dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:56.819971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1332/2644] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:56.824172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4ef85-e839-489c-8af7-e08d4d7af9a2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:56.835911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4ef85-e839-489c-8af7-e08d4d7af9a2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:56.842186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561578bc-eda3-4b19-b046-8c329ad8367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:56.842994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561578bc-eda3-4b19-b046-8c329ad8367d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:56.843407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561578bc-eda3-4b19-b046-8c329ad8367d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:56.843810 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561578bc-eda3-4b19-b046-8c329ad8367d admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:56.844730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561578bc-eda3-4b19-b046-8c329ad8367d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:38:56.845032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561578bc-eda3-4b19-b046-8c329ad8367d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:56.846627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561578bc-eda3-4b19-b046-8c329ad8367d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:56.860745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1313/2645] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:38:56.865090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ace29b-5226-4823-8f8b-c258491e7dab None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:56.874925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ace29b-5226-4823-8f8b-c258491e7dab None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:56.879538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0020bd-9dea-411c-8fed-b97ed527b2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:56.880188 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0020bd-9dea-411c-8fed-b97ed527b2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:56.880475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0020bd-9dea-411c-8fed-b97ed527b2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:56.880731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0020bd-9dea-411c-8fed-b97ed527b2b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:56.881351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0020bd-9dea-411c-8fed-b97ed527b2b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:56.883241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0020bd-9dea-411c-8fed-b97ed527b2b5 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:56.910118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1333/2646] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:38:56 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:56.914362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561578bc-eda3-4b19-b046-8c329ad8367d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:56.925230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561578bc-eda3-4b19-b046-8c329ad8367d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:56.959297 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:56.992301 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6a7d3-1a86-496e-be45-42e4507bc963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:56.992906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6a7d3-1a86-496e-be45-42e4507bc963 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:56.993182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6a7d3-1a86-496e-be45-42e4507bc963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:56.993461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6a7d3-1a86-496e-be45-42e4507bc963 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:56.994098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6a7d3-1a86-496e-be45-42e4507bc963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:56.996000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6a7d3-1a86-496e-be45-42e4507bc963 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:57.018712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1314/2647] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:38:56 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:57.022883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0020bd-9dea-411c-8fed-b97ed527b2b5 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:57.033422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0020bd-9dea-411c-8fed-b97ed527b2b5 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:57.038054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb7bef-a0a1-416d-bc68-bfa3aa07bd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:57.038651 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb7bef-a0a1-416d-bc68-bfa3aa07bd34 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.038936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb7bef-a0a1-416d-bc68-bfa3aa07bd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.039217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb7bef-a0a1-416d-bc68-bfa3aa07bd34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.039875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb7bef-a0a1-416d-bc68-bfa3aa07bd34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:57.041641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb7bef-a0a1-416d-bc68-bfa3aa07bd34 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:57.053021 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1334/2648] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:38:57.057254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6a7d3-1a86-496e-be45-42e4507bc963 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:57.067968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6a7d3-1a86-496e-be45-42e4507bc963 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:57.072980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a463de-0125-4d6f-9f35-23a34a597da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:57.073697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a463de-0125-4d6f-9f35-23a34a597da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.073988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a463de-0125-4d6f-9f35-23a34a597da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.074396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a463de-0125-4d6f-9f35-23a34a597da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.074990 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a463de-0125-4d6f-9f35-23a34a597da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:57.077229 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a463de-0125-4d6f-9f35-23a34a597da2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:57.089137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1315/2649] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:38:57.093427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb7bef-a0a1-416d-bc68-bfa3aa07bd34 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:57.103782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb7bef-a0a1-416d-bc68-bfa3aa07bd34 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:57.109953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd9804-9d44-4fe8-a964-53cddbcedf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:57.110609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd9804-9d44-4fe8-a964-53cddbcedf70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.110892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd9804-9d44-4fe8-a964-53cddbcedf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.111171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd9804-9d44-4fe8-a964-53cddbcedf70 admin admin] PATH_INFO: `/v3/projects/1d85d4c07a53428c81a4dcadcbcef12d/users/fe270e187ffd4c7a8126b8f86bcf669f/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.126496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd9804-9d44-4fe8-a964-53cddbcedf70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d85d4c07a53428c81a4dcadcbcef12d, user_id=fe270e187ffd4c7a8126b8f86bcf669f, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:57.129461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd9804-9d44-4fe8-a964-53cddbcedf70 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:57.158688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1335/2650] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:38:57 2022] PUT /identity/v3/projects/1d85d4c07a53428c81a4dcadcbcef12d/users/fe270e187ffd4c7a8126b8f86bcf669f/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:57.170466 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6a238-0855-41ef-ad97-ebc2e220a8f6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.170837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6a238-0855-41ef-ad97-ebc2e220a8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.171181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6a238-0855-41ef-ad97-ebc2e220a8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.203023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-59f6a238-0855-41ef-ad97-ebc2e220a8f6 None None] MFA Rules not processed for user `fe270e187ffd4c7a8126b8f86bcf669f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:57.204453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-59f6a238-0855-41ef-ad97-ebc2e220a8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:57.282482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1316/2651] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:57.287168 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd9804-9d44-4fe8-a964-53cddbcedf70 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:57.350401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd9804-9d44-4fe8-a964-53cddbcedf70 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:57.355108 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57d182-e982-493a-9c58-c0851f49f19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:57.355710 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57d182-e982-493a-9c58-c0851f49f19c admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.355993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57d182-e982-493a-9c58-c0851f49f19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.356817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57d182-e982-493a-9c58-c0851f49f19c admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.357453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57d182-e982-493a-9c58-c0851f49f19c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:57.359359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57d182-e982-493a-9c58-c0851f49f19c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:57.385601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1336/2652] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:38:57 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:57.389822 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6a238-0855-41ef-ad97-ebc2e220a8f6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:57.400869 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6a238-0855-41ef-ad97-ebc2e220a8f6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:57.457092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06561e-542f-4414-900a-6ba0f9513c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:57.457692 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06561e-542f-4414-900a-6ba0f9513c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.457980 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06561e-542f-4414-900a-6ba0f9513c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.458249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06561e-542f-4414-900a-6ba0f9513c98 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.458874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06561e-542f-4414-900a-6ba0f9513c98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:57.460814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06561e-542f-4414-900a-6ba0f9513c98 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:57.481005 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1317/2653] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:38:57 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:38:57.486481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57d182-e982-493a-9c58-c0851f49f19c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:57.498643 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57d182-e982-493a-9c58-c0851f49f19c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:57.503270 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a0231-d389-4e81-9bd0-d87197c38555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:57.503862 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a0231-d389-4e81-9bd0-d87197c38555 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.504185 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a0231-d389-4e81-9bd0-d87197c38555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.504490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a0231-d389-4e81-9bd0-d87197c38555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.505132 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a0231-d389-4e81-9bd0-d87197c38555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:57.507314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a0231-d389-4e81-9bd0-d87197c38555 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:57.518838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1337/2654] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:38:57.523029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06561e-542f-4414-900a-6ba0f9513c98 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:57.532943 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06561e-542f-4414-900a-6ba0f9513c98 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:57.537680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef54d5-2199-450c-a7e9-01ce1080575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:57.538309 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef54d5-2199-450c-a7e9-01ce1080575d admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.538586 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef54d5-2199-450c-a7e9-01ce1080575d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.538855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef54d5-2199-450c-a7e9-01ce1080575d admin admin] PATH_INFO: `/v3/projects/45ecafb5114a4c6db99f65508f3e6c01/users/3704214169e24df9b65fb1d4b2370dc7/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.553697 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef54d5-2199-450c-a7e9-01ce1080575d admin admin] RBAC: Authorizing `identity:create_grant(project_id=45ecafb5114a4c6db99f65508f3e6c01, user_id=3704214169e24df9b65fb1d4b2370dc7, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:57.556385 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef54d5-2199-450c-a7e9-01ce1080575d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:57.569919 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1318/2655] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:38:57 2022] PUT /identity/v3/projects/45ecafb5114a4c6db99f65508f3e6c01/users/3704214169e24df9b65fb1d4b2370dc7/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:57.574153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a0231-d389-4e81-9bd0-d87197c38555 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:57.584917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a0231-d389-4e81-9bd0-d87197c38555 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:57.605045 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:57.645003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10e78b-4f7a-4d75-b5a3-483195f30024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:57.645609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10e78b-4f7a-4d75-b5a3-483195f30024 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.645892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10e78b-4f7a-4d75-b5a3-483195f30024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.646166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10e78b-4f7a-4d75-b5a3-483195f30024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.646810 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10e78b-4f7a-4d75-b5a3-483195f30024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:57.648452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10e78b-4f7a-4d75-b5a3-483195f30024 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:57.658672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1338/2656] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:38:57.662672 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef54d5-2199-450c-a7e9-01ce1080575d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:57.668135 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:57.675333 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef54d5-2199-450c-a7e9-01ce1080575d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:57.680124 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93853959-54ca-48ce-bda5-272f3efd8267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:57.680772 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93853959-54ca-48ce-bda5-272f3efd8267 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.681047 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93853959-54ca-48ce-bda5-272f3efd8267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.681522 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93853959-54ca-48ce-bda5-272f3efd8267 admin admin] PATH_INFO: `/v3/projects/45ecafb5114a4c6db99f65508f3e6c01/users/3704214169e24df9b65fb1d4b2370dc7/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.683288 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93853959-54ca-48ce-bda5-272f3efd8267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45ecafb5114a4c6db99f65508f3e6c01, user_id=3704214169e24df9b65fb1d4b2370dc7, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:57.686461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93853959-54ca-48ce-bda5-272f3efd8267 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:57.697393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1319/2657] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:38:57 2022] PUT /identity/v3/projects/45ecafb5114a4c6db99f65508f3e6c01/users/3704214169e24df9b65fb1d4b2370dc7/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:38:57.708491 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62650-9f5d-4061-962c-65ea56367b8c None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.708786 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62650-9f5d-4061-962c-65ea56367b8c None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.709064 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62650-9f5d-4061-962c-65ea56367b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.736261 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-3cc62650-9f5d-4061-962c-65ea56367b8c None None] MFA Rules not processed for user `3704214169e24df9b65fb1d4b2370dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:38:57.737509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3cc62650-9f5d-4061-962c-65ea56367b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:57.806521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1339/2658] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:38:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:38:57.819390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93853959-54ca-48ce-bda5-272f3efd8267 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:57.821228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93853959-54ca-48ce-bda5-272f3efd8267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:57.879551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93853959-54ca-48ce-bda5-272f3efd8267 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:57.883175 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d273d-ef5b-45d9-8de4-be793930a63a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:57.883776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d273d-ef5b-45d9-8de4-be793930a63a service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.884063 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d273d-ef5b-45d9-8de4-be793930a63a service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.884343 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d273d-ef5b-45d9-8de4-be793930a63a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:57.894096 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d273d-ef5b-45d9-8de4-be793930a63a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:57.895521 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d273d-ef5b-45d9-8de4-be793930a63a service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:57.905091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1320/2659] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:38:57.948684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc62650-9f5d-4061-962c-65ea56367b8c None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:38:57.950681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3cc62650-9f5d-4061-962c-65ea56367b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:38:57.962396 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc62650-9f5d-4061-962c-65ea56367b8c None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:38:57.965957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bea310-d243-4c95-899a-a34a729ba453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:38:57.966586 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea310-d243-4c95-899a-a34a729ba453 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:38:57.966864 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea310-d243-4c95-899a-a34a729ba453 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:38:57.967131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea310-d243-4c95-899a-a34a729ba453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:38:58.022204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bea310-d243-4c95-899a-a34a729ba453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:38:58.023750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bea310-d243-4c95-899a-a34a729ba453 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:38:58.032833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1340/2660] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:38:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:01.308811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d273d-ef5b-45d9-8de4-be793930a63a service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:01.374089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d273d-ef5b-45d9-8de4-be793930a63a service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:01.379419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:01.380058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:01.380350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:01.380625 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] PATH_INFO: `/v3/users/fe270e187ffd4c7a8126b8f86bcf669f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:01.381744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe270e187ffd4c7a8126b8f86bcf669f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:01.383667 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:01.429366 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe270e187ffd4c7a8126b8f86bcf669f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:01.430009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe270e187ffd4c7a8126b8f86bcf669f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:01.430602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe270e187ffd4c7a8126b8f86bcf669f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:01.438634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe270e187ffd4c7a8126b8f86bcf669f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:01.446002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe270e187ffd4c7a8126b8f86bcf669f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:01.463416 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1321/2661] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:39:01 2022] DELETE /identity/v3/users/fe270e187ffd4c7a8126b8f86bcf669f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:01.468016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4754ba8-da98-46ec-9a99-5417f0563bb4 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:01.468528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4754ba8-da98-46ec-9a99-5417f0563bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:01.468654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4754ba8-da98-46ec-9a99-5417f0563bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:01.488075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f4754ba8-da98-46ec-9a99-5417f0563bb4 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:01.489352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f4754ba8-da98-46ec-9a99-5417f0563bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:01.530429 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:01.551255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1341/2662] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:01.561715 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:01.563220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:01.581517 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:01.628418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49892fc-27b5-4d2a-8ab2-6457f58451d8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:01.632016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1923b4-a2df-4673-aa49-81cac6bd89a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:01.632660 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1923b4-a2df-4673-aa49-81cac6bd89a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:01.632941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1923b4-a2df-4673-aa49-81cac6bd89a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:01.633214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1923b4-a2df-4673-aa49-81cac6bd89a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:01.642708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1923b4-a2df-4673-aa49-81cac6bd89a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:01.644157 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1923b4-a2df-4673-aa49-81cac6bd89a5 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:01.653496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1322/2663] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:39:01.932860 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4754ba8-da98-46ec-9a99-5417f0563bb4 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:01.934434 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f4754ba8-da98-46ec-9a99-5417f0563bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:01.947091 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4754ba8-da98-46ec-9a99-5417f0563bb4 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:01.951683 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:01.952426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:01.952769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:01.953104 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] PATH_INFO: `/v3/projects/1d85d4c07a53428c81a4dcadcbcef12d` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:01.954290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d85d4c07a53428c81a4dcadcbcef12d)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:01.956362 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:02.004345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d85d4c07a53428c81a4dcadcbcef12d', 'request_id': 'req-d5273f6b-3418-49b7-91a6-19e8da8865e8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.010428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d85d4c07a53428c81a4dcadcbcef12d', 'request_id': 'req-d5273f6b-3418-49b7-91a6-19e8da8865e8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.019733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d85d4c07a53428c81a4dcadcbcef12d', 'request_id': 'req-d5273f6b-3418-49b7-91a6-19e8da8865e8'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.034382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] The token cache is being invalidate because project 1d85d4c07a53428c81a4dcadcbcef12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:39:02.035255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.036769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.038604 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1342/2664] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:39:01 2022] DELETE /identity/v3/projects/1d85d4c07a53428c81a4dcadcbcef12d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:02.042588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1923b4-a2df-4673-aa49-81cac6bd89a5 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:02.052730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1923b4-a2df-4673-aa49-81cac6bd89a5 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:02.109059 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73607dc-ff44-4cd5-8c62-67ef9c61e8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:02.109680 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73607dc-ff44-4cd5-8c62-67ef9c61e8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:02.109959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73607dc-ff44-4cd5-8c62-67ef9c61e8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:02.110231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73607dc-ff44-4cd5-8c62-67ef9c61e8db admin admin] PATH_INFO: `/v3/users/3704214169e24df9b65fb1d4b2370dc7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:02.111341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73607dc-ff44-4cd5-8c62-67ef9c61e8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=3704214169e24df9b65fb1d4b2370dc7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:02.113271 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73607dc-ff44-4cd5-8c62-67ef9c61e8db admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:02.160816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f73607dc-ff44-4cd5-8c62-67ef9c61e8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3704214169e24df9b65fb1d4b2370dc7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.161470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f73607dc-ff44-4cd5-8c62-67ef9c61e8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3704214169e24df9b65fb1d4b2370dc7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.162089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f73607dc-ff44-4cd5-8c62-67ef9c61e8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3704214169e24df9b65fb1d4b2370dc7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.169721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f73607dc-ff44-4cd5-8c62-67ef9c61e8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3704214169e24df9b65fb1d4b2370dc7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.176851 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f73607dc-ff44-4cd5-8c62-67ef9c61e8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3704214169e24df9b65fb1d4b2370dc7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.194062 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1323/2665] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:39:02 2022] DELETE /identity/v3/users/3704214169e24df9b65fb1d4b2370dc7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:02.469833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:02.471350 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:02.485576 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5273f6b-3418-49b7-91a6-19e8da8865e8 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:02.546220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:02.546849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:02.547126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:02.547398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] PATH_INFO: `/v3/projects/45ecafb5114a4c6db99f65508f3e6c01` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:02.548550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=45ecafb5114a4c6db99f65508f3e6c01)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:02.550560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:02.601942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45ecafb5114a4c6db99f65508f3e6c01', 'request_id': 'req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.608273 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45ecafb5114a4c6db99f65508f3e6c01', 'request_id': 'req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.613773 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45ecafb5114a4c6db99f65508f3e6c01', 'request_id': 'req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.625019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:02.630385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] The token cache is being invalidate because project 45ecafb5114a4c6db99f65508f3e6c01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:39:02.631163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.632067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-821f66ce-c5a2-48f1-b9d9-30e003e1cc0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:02.633935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1343/2666] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:39:02 2022] DELETE /identity/v3/projects/45ecafb5114a4c6db99f65508f3e6c01 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:02.640454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef9128-eeef-4361-84aa-7af8e77aee4b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:02.640998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef9128-eeef-4361-84aa-7af8e77aee4b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:02.640998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef9128-eeef-4361-84aa-7af8e77aee4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:02.661249 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8cef9128-eeef-4361-84aa-7af8e77aee4b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:02.662529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8cef9128-eeef-4361-84aa-7af8e77aee4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:02.692808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:02.725994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1324/2667] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:02.734676 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf5534-b301-4c3c-bbc3-1c24e835de14 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:02.734961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf5534-b301-4c3c-bbc3-1c24e835de14 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:02.735239 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf5534-b301-4c3c-bbc3-1c24e835de14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:02.756810 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-71bf5534-b301-4c3c-bbc3-1c24e835de14 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:02.758266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-71bf5534-b301-4c3c-bbc3-1c24e835de14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:02.767195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1344/2668] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:02.771579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef9128-eeef-4361-84aa-7af8e77aee4b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:02.782160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef9128-eeef-4361-84aa-7af8e77aee4b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:02.787251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afb621-9705-441f-a1af-ec728033c68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:02.787859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afb621-9705-441f-a1af-ec728033c68f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:02.788179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afb621-9705-441f-a1af-ec728033c68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:02.788453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afb621-9705-441f-a1af-ec728033c68f admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:02.789107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afb621-9705-441f-a1af-ec728033c68f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:39:02.789508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afb621-9705-441f-a1af-ec728033c68f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:02.791186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afb621-9705-441f-a1af-ec728033c68f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:02.804491 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1325/2669] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:39:02.808731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf5534-b301-4c3c-bbc3-1c24e835de14 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:02.819650 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf5534-b301-4c3c-bbc3-1c24e835de14 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:02.827701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f738094-c201-4eef-9411-21d84b816014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:02.828541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f738094-c201-4eef-9411-21d84b816014 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:02.828825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f738094-c201-4eef-9411-21d84b816014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:02.829159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f738094-c201-4eef-9411-21d84b816014 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:02.829738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f738094-c201-4eef-9411-21d84b816014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:02.831767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f738094-c201-4eef-9411-21d84b816014 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:02.863623 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1345/2670] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:39:02 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:39:02.868985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afb621-9705-441f-a1af-ec728033c68f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:02.889870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afb621-9705-441f-a1af-ec728033c68f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:02.963254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2031de-7a7f-4726-9f3f-60b13f30220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:02.963254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2031de-7a7f-4726-9f3f-60b13f30220d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:02.963771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2031de-7a7f-4726-9f3f-60b13f30220d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:02.963933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2031de-7a7f-4726-9f3f-60b13f30220d admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:02.964730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2031de-7a7f-4726-9f3f-60b13f30220d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:02.967235 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2031de-7a7f-4726-9f3f-60b13f30220d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:02.994855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1326/2671] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:39:02 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:39:02.996089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f738094-c201-4eef-9411-21d84b816014 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:03.006626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f738094-c201-4eef-9411-21d84b816014 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:03.011900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89839f59-3da4-4bf0-a0f0-71747c8cbd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:03.012531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89839f59-3da4-4bf0-a0f0-71747c8cbd3e admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.012838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89839f59-3da4-4bf0-a0f0-71747c8cbd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.013114 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89839f59-3da4-4bf0-a0f0-71747c8cbd3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.013755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89839f59-3da4-4bf0-a0f0-71747c8cbd3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:03.015550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89839f59-3da4-4bf0-a0f0-71747c8cbd3e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:03.026041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1346/2672] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:39:03.030735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2031de-7a7f-4726-9f3f-60b13f30220d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:03.041126 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2031de-7a7f-4726-9f3f-60b13f30220d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:03.046390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df848d-cd25-499c-b003-d38ff38bacbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:03.046977 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df848d-cd25-499c-b003-d38ff38bacbd admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.047257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df848d-cd25-499c-b003-d38ff38bacbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.047527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df848d-cd25-499c-b003-d38ff38bacbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.048208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df848d-cd25-499c-b003-d38ff38bacbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:03.049799 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df848d-cd25-499c-b003-d38ff38bacbd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:03.061014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1327/2673] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:39:03.065324 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89839f59-3da4-4bf0-a0f0-71747c8cbd3e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:03.075314 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89839f59-3da4-4bf0-a0f0-71747c8cbd3e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:03.080097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6c975-5cd5-4f61-822a-192d9dbd60be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:03.080743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6c975-5cd5-4f61-822a-192d9dbd60be admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.081026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6c975-5cd5-4f61-822a-192d9dbd60be admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.081300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6c975-5cd5-4f61-822a-192d9dbd60be admin admin] PATH_INFO: `/v3/projects/0e364cccd0884588895a87d55035cfb6/users/4eba13e2709f410fab4b6564a2d4430e/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.096353 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6c975-5cd5-4f61-822a-192d9dbd60be admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e364cccd0884588895a87d55035cfb6, user_id=4eba13e2709f410fab4b6564a2d4430e, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:03.099071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6c975-5cd5-4f61-822a-192d9dbd60be admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:03.112738 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1347/2674] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:39:03 2022] PUT /identity/v3/projects/0e364cccd0884588895a87d55035cfb6/users/4eba13e2709f410fab4b6564a2d4430e/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:03.124174 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c089f-cd02-41c3-8f03-877b1608b2db None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.124348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c089f-cd02-41c3-8f03-877b1608b2db None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.124628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c089f-cd02-41c3-8f03-877b1608b2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.151963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-651c089f-cd02-41c3-8f03-877b1608b2db None None] MFA Rules not processed for user `4eba13e2709f410fab4b6564a2d4430e`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:03.153203 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-651c089f-cd02-41c3-8f03-877b1608b2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:03.221125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1328/2675] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:03.225628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6c975-5cd5-4f61-822a-192d9dbd60be admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:03.288363 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6c975-5cd5-4f61-822a-192d9dbd60be admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:03.293348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e78d01-faf7-480e-bcc6-1f4b1176c37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:03.293947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e78d01-faf7-480e-bcc6-1f4b1176c37d admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.294262 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e78d01-faf7-480e-bcc6-1f4b1176c37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.294539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e78d01-faf7-480e-bcc6-1f4b1176c37d admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.295179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e78d01-faf7-480e-bcc6-1f4b1176c37d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:03.297130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e78d01-faf7-480e-bcc6-1f4b1176c37d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:03.324524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1348/2676] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:39:03 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:39:03.329236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c089f-cd02-41c3-8f03-877b1608b2db None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:03.340634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c089f-cd02-41c3-8f03-877b1608b2db None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:03.369451 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:03.401351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c2048-464a-4e21-b328-efdb4479e007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:03.401890 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c2048-464a-4e21-b328-efdb4479e007 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.402173 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c2048-464a-4e21-b328-efdb4479e007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.402446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c2048-464a-4e21-b328-efdb4479e007 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.403081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c2048-464a-4e21-b328-efdb4479e007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:03.405635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c2048-464a-4e21-b328-efdb4479e007 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:03.427742 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1329/2677] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:39:03 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:39:03.431659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e78d01-faf7-480e-bcc6-1f4b1176c37d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:03.441938 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e78d01-faf7-480e-bcc6-1f4b1176c37d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:03.446666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab9b15-8a1b-43d8-be95-678bfc7bc350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:03.447257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab9b15-8a1b-43d8-be95-678bfc7bc350 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.447539 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab9b15-8a1b-43d8-be95-678bfc7bc350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.447813 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab9b15-8a1b-43d8-be95-678bfc7bc350 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.448827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab9b15-8a1b-43d8-be95-678bfc7bc350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:03.450960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab9b15-8a1b-43d8-be95-678bfc7bc350 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:03.461723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1349/2678] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:39:03.467430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c2048-464a-4e21-b328-efdb4479e007 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:03.480097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c2048-464a-4e21-b328-efdb4479e007 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:03.484850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466c104-3709-43d7-8c69-f200f6c35008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:03.485498 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466c104-3709-43d7-8c69-f200f6c35008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.485778 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466c104-3709-43d7-8c69-f200f6c35008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.486127 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466c104-3709-43d7-8c69-f200f6c35008 admin admin] PATH_INFO: `/v3/projects/37a7506a538049899d3e410598cd68c2/users/f72aa11018fd40d29a9ae9da53bdc3f9/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.502865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466c104-3709-43d7-8c69-f200f6c35008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a7506a538049899d3e410598cd68c2, user_id=f72aa11018fd40d29a9ae9da53bdc3f9, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:03.505571 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466c104-3709-43d7-8c69-f200f6c35008 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:03.519194 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1330/2679] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:39:03 2022] PUT /identity/v3/projects/37a7506a538049899d3e410598cd68c2/users/f72aa11018fd40d29a9ae9da53bdc3f9/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:03.532084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab9b15-8a1b-43d8-be95-678bfc7bc350 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:03.542954 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab9b15-8a1b-43d8-be95-678bfc7bc350 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:03.601199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4e756-4179-4df6-9b72-05070cce45ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:03.601797 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4e756-4179-4df6-9b72-05070cce45ac admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.602073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4e756-4179-4df6-9b72-05070cce45ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.602348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4e756-4179-4df6-9b72-05070cce45ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.602984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4e756-4179-4df6-9b72-05070cce45ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:03.604612 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4e756-4179-4df6-9b72-05070cce45ac admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:03.615098 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1350/2680] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:39:03.619160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466c104-3709-43d7-8c69-f200f6c35008 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:03.631091 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466c104-3709-43d7-8c69-f200f6c35008 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:03.635572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a407a-2235-4649-8ce1-87ee2c88ef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:03.636244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a407a-2235-4649-8ce1-87ee2c88ef65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.636522 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a407a-2235-4649-8ce1-87ee2c88ef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.636792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a407a-2235-4649-8ce1-87ee2c88ef65 admin admin] PATH_INFO: `/v3/projects/37a7506a538049899d3e410598cd68c2/users/f72aa11018fd40d29a9ae9da53bdc3f9/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.638725 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a407a-2235-4649-8ce1-87ee2c88ef65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a7506a538049899d3e410598cd68c2, user_id=f72aa11018fd40d29a9ae9da53bdc3f9, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:03.641368 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a407a-2235-4649-8ce1-87ee2c88ef65 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:03.651269 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1331/2681] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:39:03 2022] PUT /identity/v3/projects/37a7506a538049899d3e410598cd68c2/users/f72aa11018fd40d29a9ae9da53bdc3f9/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:03.662116 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea14bd-7a76-474f-864e-24911fa1dfbf None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.662416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea14bd-7a76-474f-864e-24911fa1dfbf None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.662751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea14bd-7a76-474f-864e-24911fa1dfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.671698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:03.691186 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-5dea14bd-7a76-474f-864e-24911fa1dfbf None None] MFA Rules not processed for user `f72aa11018fd40d29a9ae9da53bdc3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:03.692410 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-5dea14bd-7a76-474f-864e-24911fa1dfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:03.761303 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1351/2682] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:03.772677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a407a-2235-4649-8ce1-87ee2c88ef65 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:03.774417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6e8a407a-2235-4649-8ce1-87ee2c88ef65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:03.836102 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a407a-2235-4649-8ce1-87ee2c88ef65 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:03.839803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ddcfc-534f-4810-91f8-b9707e75f868 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:03.840447 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ddcfc-534f-4810-91f8-b9707e75f868 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.840721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ddcfc-534f-4810-91f8-b9707e75f868 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.841480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ddcfc-534f-4810-91f8-b9707e75f868 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:03.850531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ddcfc-534f-4810-91f8-b9707e75f868 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:03.851971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ddcfc-534f-4810-91f8-b9707e75f868 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:03.863954 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1332/2683] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:03.945019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea14bd-7a76-474f-864e-24911fa1dfbf None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:03.954920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea14bd-7a76-474f-864e-24911fa1dfbf None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:03.958596 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d6c08-d6b0-4e31-a194-741acb5e08a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:03.959189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d6c08-d6b0-4e31-a194-741acb5e08a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:03.959466 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d6c08-d6b0-4e31-a194-741acb5e08a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:03.959764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d6c08-d6b0-4e31-a194-741acb5e08a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:04.025516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d6c08-d6b0-4e31-a194-741acb5e08a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:04.027015 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d6c08-d6b0-4e31-a194-741acb5e08a4 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:04.037267 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1352/2684] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:04.328352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ddcfc-534f-4810-91f8-b9707e75f868 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:04.394431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ddcfc-534f-4810-91f8-b9707e75f868 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:04.399811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:04.400480 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:04.400757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:04.401029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] PATH_INFO: `/v3/users/4eba13e2709f410fab4b6564a2d4430e` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:04.402183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eba13e2709f410fab4b6564a2d4430e)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:04.404121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:04.438731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:04.460914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eba13e2709f410fab4b6564a2d4430e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:04.462033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eba13e2709f410fab4b6564a2d4430e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:04.462766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eba13e2709f410fab4b6564a2d4430e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:04.471940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eba13e2709f410fab4b6564a2d4430e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:04.480284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eba13e2709f410fab4b6564a2d4430e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:04.500750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1333/2685] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:39:04 2022] DELETE /identity/v3/users/4eba13e2709f410fab4b6564a2d4430e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:04.505370 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4072c07-d6d1-4d6e-a7fe-9812fe227428 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:04.505665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4072c07-d6d1-4d6e-a7fe-9812fe227428 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:04.505951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4072c07-d6d1-4d6e-a7fe-9812fe227428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:04.527493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-f4072c07-d6d1-4d6e-a7fe-9812fe227428 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:04.529124 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f4072c07-d6d1-4d6e-a7fe-9812fe227428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:04.592657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1353/2686] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:04.602793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:04.604251 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:04.665800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7aedb8-acc4-48b1-a71c-c2b4f7ec1a35 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:04.669753 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3295df-ff87-4973-9408-fca88b5907cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:04.670387 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3295df-ff87-4973-9408-fca88b5907cf service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:04.670708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3295df-ff87-4973-9408-fca88b5907cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:04.670983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3295df-ff87-4973-9408-fca88b5907cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:04.680800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3295df-ff87-4973-9408-fca88b5907cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:04.682241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3295df-ff87-4973-9408-fca88b5907cf service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:04.692408 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1334/2687] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:04.968366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4072c07-d6d1-4d6e-a7fe-9812fe227428 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:04.969785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-f4072c07-d6d1-4d6e-a7fe-9812fe227428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:04.982964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4072c07-d6d1-4d6e-a7fe-9812fe227428 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:04.987953 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:04.988621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:04.988906 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:04.989192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] PATH_INFO: `/v3/projects/0e364cccd0884588895a87d55035cfb6` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:04.990374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e364cccd0884588895a87d55035cfb6)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:04.992338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:05.017835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:05.040425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e364cccd0884588895a87d55035cfb6', 'request_id': 'req-cf7cb805-377b-4733-a7e2-316b85a327ef'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.047035 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e364cccd0884588895a87d55035cfb6', 'request_id': 'req-cf7cb805-377b-4733-a7e2-316b85a327ef'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.052490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e364cccd0884588895a87d55035cfb6', 'request_id': 'req-cf7cb805-377b-4733-a7e2-316b85a327ef'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.066090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] The token cache is being invalidate because project 0e364cccd0884588895a87d55035cfb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:39:05.066396 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.067029 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.068597 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1354/2688] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:39:04 2022] DELETE /identity/v3/projects/0e364cccd0884588895a87d55035cfb6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:05.074147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3295df-ff87-4973-9408-fca88b5907cf service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:05.085632 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3295df-ff87-4973-9408-fca88b5907cf service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:05.143185 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8786e-1745-432a-a77b-6693918feda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:05.143809 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8786e-1745-432a-a77b-6693918feda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:05.144125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8786e-1745-432a-a77b-6693918feda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:05.144410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8786e-1745-432a-a77b-6693918feda5 admin admin] PATH_INFO: `/v3/users/f72aa11018fd40d29a9ae9da53bdc3f9` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:05.145530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8786e-1745-432a-a77b-6693918feda5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72aa11018fd40d29a9ae9da53bdc3f9)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:05.147449 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8786e-1745-432a-a77b-6693918feda5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:05.194592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49c8786e-1745-432a-a77b-6693918feda5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72aa11018fd40d29a9ae9da53bdc3f9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.195246 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49c8786e-1745-432a-a77b-6693918feda5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72aa11018fd40d29a9ae9da53bdc3f9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.195857 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49c8786e-1745-432a-a77b-6693918feda5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72aa11018fd40d29a9ae9da53bdc3f9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.197536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:05.204284 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49c8786e-1745-432a-a77b-6693918feda5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72aa11018fd40d29a9ae9da53bdc3f9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.212225 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-49c8786e-1745-432a-a77b-6693918feda5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72aa11018fd40d29a9ae9da53bdc3f9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.230386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1335/2689] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:39:05 2022] DELETE /identity/v3/users/f72aa11018fd40d29a9ae9da53bdc3f9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:39:05.535756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:05.537358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:05.551872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cb805-377b-4733-a7e2-316b85a327ef admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:05.613142 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:05.613818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:05.614052 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:05.614331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] PATH_INFO: `/v3/projects/37a7506a538049899d3e410598cd68c2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:05.615450 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a7506a538049899d3e410598cd68c2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:05.617440 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:05.664768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a7506a538049899d3e410598cd68c2', 'request_id': 'req-1f66a593-8e97-4642-9882-9ebaa183b983'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.670787 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a7506a538049899d3e410598cd68c2', 'request_id': 'req-1f66a593-8e97-4642-9882-9ebaa183b983'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.676159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a7506a538049899d3e410598cd68c2', 'request_id': 'req-1f66a593-8e97-4642-9882-9ebaa183b983'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.688550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] The token cache is being invalidate because project 37a7506a538049899d3e410598cd68c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:39:05.688844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.689469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1f66a593-8e97-4642-9882-9ebaa183b983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:05.690956 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1355/2690] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:39:05 2022] DELETE /identity/v3/projects/37a7506a538049899d3e410598cd68c2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:05.695985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877de8c-42e2-4d02-93c7-03dfd053474a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:05.696710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877de8c-42e2-4d02-93c7-03dfd053474a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:05.696710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877de8c-42e2-4d02-93c7-03dfd053474a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:05.717903 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6877de8c-42e2-4d02-93c7-03dfd053474a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:05.719165 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6877de8c-42e2-4d02-93c7-03dfd053474a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:05.778391 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1336/2691] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:05.786907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b4eee-cc30-48b1-a5d8-c804c7972d9a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:05.787198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b4eee-cc30-48b1-a5d8-c804c7972d9a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:05.787477 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b4eee-cc30-48b1-a5d8-c804c7972d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:05.808665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-365b4eee-cc30-48b1-a5d8-c804c7972d9a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:05.810767 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-365b4eee-cc30-48b1-a5d8-c804c7972d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:05.818698 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1356/2692] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:05.823100 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877de8c-42e2-4d02-93c7-03dfd053474a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:05.832864 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877de8c-42e2-4d02-93c7-03dfd053474a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:05.837482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfdbc4-2ae9-415f-8a9f-e013fe38765e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:05.838088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfdbc4-2ae9-415f-8a9f-e013fe38765e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:05.838357 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfdbc4-2ae9-415f-8a9f-e013fe38765e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:05.838634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfdbc4-2ae9-415f-8a9f-e013fe38765e admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:05.839287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cfdbc4-2ae9-415f-8a9f-e013fe38765e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:39:05.839592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cfdbc4-2ae9-415f-8a9f-e013fe38765e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:05.841255 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cfdbc4-2ae9-415f-8a9f-e013fe38765e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:05.852611 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1337/2693] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:39:05.857263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b4eee-cc30-48b1-a5d8-c804c7972d9a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:05.866655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b4eee-cc30-48b1-a5d8-c804c7972d9a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:05.871315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a55ac-be15-4d39-9501-54fe69015ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:05.871907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a55ac-be15-4d39-9501-54fe69015ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:05.872218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a55ac-be15-4d39-9501-54fe69015ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:05.873109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a55ac-be15-4d39-9501-54fe69015ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:05.873732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a55ac-be15-4d39-9501-54fe69015ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:05.875641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a55ac-be15-4d39-9501-54fe69015ff4 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:05.902311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1357/2694] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:39:05 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:39:05.906978 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfdbc4-2ae9-415f-8a9f-e013fe38765e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:05.917160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfdbc4-2ae9-415f-8a9f-e013fe38765e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:05.972659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e9ea5-e2a0-4674-b8cd-13e022c05896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:05.973177 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e9ea5-e2a0-4674-b8cd-13e022c05896 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:05.973457 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e9ea5-e2a0-4674-b8cd-13e022c05896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:05.973733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e9ea5-e2a0-4674-b8cd-13e022c05896 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:05.974353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e9ea5-e2a0-4674-b8cd-13e022c05896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:05.976329 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e9ea5-e2a0-4674-b8cd-13e022c05896 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:05.997789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1338/2695] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:39:05 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:39:06.002038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a55ac-be15-4d39-9501-54fe69015ff4 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:06.012785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a55ac-be15-4d39-9501-54fe69015ff4 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:06.017437 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16abe90-6f26-45e5-80a7-df9940e5bc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:06.018032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16abe90-6f26-45e5-80a7-df9940e5bc62 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:06.018316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16abe90-6f26-45e5-80a7-df9940e5bc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:06.018589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16abe90-6f26-45e5-80a7-df9940e5bc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:06.019234 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16abe90-6f26-45e5-80a7-df9940e5bc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:06.020886 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16abe90-6f26-45e5-80a7-df9940e5bc62 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:06.031403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1358/2696] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:39:06.035476 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e9ea5-e2a0-4674-b8cd-13e022c05896 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:06.046341 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e9ea5-e2a0-4674-b8cd-13e022c05896 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:06.051211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75177b17-215d-48b1-a705-8fc2d8669eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:06.051804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75177b17-215d-48b1-a705-8fc2d8669eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:06.052116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75177b17-215d-48b1-a705-8fc2d8669eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:06.052395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75177b17-215d-48b1-a705-8fc2d8669eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:06.053032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75177b17-215d-48b1-a705-8fc2d8669eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:06.054621 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75177b17-215d-48b1-a705-8fc2d8669eb0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:06.065925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1339/2697] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:39:06.069975 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16abe90-6f26-45e5-80a7-df9940e5bc62 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:06.075456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:06.081593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16abe90-6f26-45e5-80a7-df9940e5bc62 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:06.087802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a1c04-d98a-4692-b76d-7dc3e327a804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:06.088679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a1c04-d98a-4692-b76d-7dc3e327a804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:06.089027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a1c04-d98a-4692-b76d-7dc3e327a804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:06.089476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a1c04-d98a-4692-b76d-7dc3e327a804 admin admin] PATH_INFO: `/v3/projects/d896e5cd66cb46b4863d11c9492bd72e/users/b434694b8bc3452fbbf43e65f7626666/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:06.103978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a1c04-d98a-4692-b76d-7dc3e327a804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d896e5cd66cb46b4863d11c9492bd72e, user_id=b434694b8bc3452fbbf43e65f7626666, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:06.106690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a1c04-d98a-4692-b76d-7dc3e327a804 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:06.119089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1359/2698] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:39:06 2022] PUT /identity/v3/projects/d896e5cd66cb46b4863d11c9492bd72e/users/b434694b8bc3452fbbf43e65f7626666/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:06.130890 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657bb2-d058-48d0-a3a4-4ad30b73dd55 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:06.131187 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657bb2-d058-48d0-a3a4-4ad30b73dd55 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:06.131466 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657bb2-d058-48d0-a3a4-4ad30b73dd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:06.163166 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d9657bb2-d058-48d0-a3a4-4ad30b73dd55 None None] MFA Rules not processed for user `b434694b8bc3452fbbf43e65f7626666`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:06.164443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d9657bb2-d058-48d0-a3a4-4ad30b73dd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:06.183026 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:06.230669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1340/2699] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:06.242508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a1c04-d98a-4692-b76d-7dc3e327a804 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:06.244609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-858a1c04-d98a-4692-b76d-7dc3e327a804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:06.325475 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a1c04-d98a-4692-b76d-7dc3e327a804 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:06.329219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26c513-4eea-494d-8ec3-f0979a71d15a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '794dbf38682e4b4f939854eecbacaf07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:06.329817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26c513-4eea-494d-8ec3-f0979a71d15a service glance] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:06.330104 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26c513-4eea-494d-8ec3-f0979a71d15a service glance] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:06.330448 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26c513-4eea-494d-8ec3-f0979a71d15a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:06.338976 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26c513-4eea-494d-8ec3-f0979a71d15a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:06.340515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26c513-4eea-494d-8ec3-f0979a71d15a service glance] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:06.350333 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1360/2700] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:16.227970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=18 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:16.346290 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=44,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:21.407479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9657bb2-d058-48d0-a3a4-4ad30b73dd55 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:21.408927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d9657bb2-d058-48d0-a3a4-4ad30b73dd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:21.466446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9657bb2-d058-48d0-a3a4-4ad30b73dd55 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:21.469937 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c6e89-318f-4ea5-9b7c-21f75d3e0b91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:21.470532 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c6e89-318f-4ea5-9b7c-21f75d3e0b91 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:21.470814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c6e89-318f-4ea5-9b7c-21f75d3e0b91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:21.471087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c6e89-318f-4ea5-9b7c-21f75d3e0b91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:21.473454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-183c6e89-318f-4ea5-9b7c-21f75d3e0b91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:21.545638 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c6e89-318f-4ea5-9b7c-21f75d3e0b91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:21.547170 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c6e89-318f-4ea5-9b7c-21f75d3e0b91 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:21.560287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1341/2701] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:22.029991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26c513-4eea-494d-8ec3-f0979a71d15a service glance] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:22.031584 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fc26c513-4eea-494d-8ec3-f0979a71d15a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:22.091375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c6e89-318f-4ea5-9b7c-21f75d3e0b91 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:22.093939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-183c6e89-318f-4ea5-9b7c-21f75d3e0b91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:22.115992 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c6e89-318f-4ea5-9b7c-21f75d3e0b91 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:22.134985 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:22.135648 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:22.135931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:22.136243 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] PATH_INFO: `/v3/users/4ce9eb7417a7482cb9cdaed16cb6aeb9` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:22.137419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ce9eb7417a7482cb9cdaed16cb6aeb9)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:22.139418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:22.217881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ce9eb7417a7482cb9cdaed16cb6aeb9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:22.218538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ce9eb7417a7482cb9cdaed16cb6aeb9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:22.219492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ce9eb7417a7482cb9cdaed16cb6aeb9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:22.230144 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ce9eb7417a7482cb9cdaed16cb6aeb9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:22.238658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ce9eb7417a7482cb9cdaed16cb6aeb9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:22.261220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1342/2702] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:39:22 2022] DELETE /identity/v3/users/4ce9eb7417a7482cb9cdaed16cb6aeb9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:39:22.265271 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26c513-4eea-494d-8ec3-f0979a71d15a service glance] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:22.330502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb1f30-4bb7-4569-b5ae-bbaef1b16a8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:22.331114 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb1f30-4bb7-4569-b5ae-bbaef1b16a8e service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:22.331393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb1f30-4bb7-4569-b5ae-bbaef1b16a8e service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:22.331665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb1f30-4bb7-4569-b5ae-bbaef1b16a8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:22.333626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d5bb1f30-4bb7-4569-b5ae-bbaef1b16a8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:22.345588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb1f30-4bb7-4569-b5ae-bbaef1b16a8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:22.347282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb1f30-4bb7-4569-b5ae-bbaef1b16a8e service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:22.358043 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1361/2703] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:22.439383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:22.440862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:22.471726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:22.515258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c0688-1abb-4539-b1b3-988bc9a61638 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:22.519923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:22.520647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:22.520959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:22.521295 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] PATH_INFO: `/v3/projects/f1eac6259f65441d9abbabe2a04afad1` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:22.522427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1eac6259f65441d9abbabe2a04afad1)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:22.524393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:22.576417 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1eac6259f65441d9abbabe2a04afad1', 'request_id': 'req-11b2bf0b-e63b-4153-933f-bf9238d97d4a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:22.582874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1eac6259f65441d9abbabe2a04afad1', 'request_id': 'req-11b2bf0b-e63b-4153-933f-bf9238d97d4a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:22.589425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1eac6259f65441d9abbabe2a04afad1', 'request_id': 'req-11b2bf0b-e63b-4153-933f-bf9238d97d4a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:22.604939 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] The token cache is being invalidate because project f1eac6259f65441d9abbabe2a04afad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:39:22.605218 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:22.605814 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-11b2bf0b-e63b-4153-933f-bf9238d97d4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:22.607611 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1343/2704] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:39:22 2022] DELETE /identity/v3/projects/f1eac6259f65441d9abbabe2a04afad1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:22.612855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502229-1d1b-4061-9aeb-70b32d80f5ee None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:22.613267 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502229-1d1b-4061-9aeb-70b32d80f5ee None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:22.614136 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502229-1d1b-4061-9aeb-70b32d80f5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:22.633684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e3502229-1d1b-4061-9aeb-70b32d80f5ee None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:22.634882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e3502229-1d1b-4061-9aeb-70b32d80f5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:22.692919 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1362/2705] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:22.700705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb981-a6ad-4896-abdd-85fe71cb199c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:22.700996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb981-a6ad-4896-abdd-85fe71cb199c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:22.701279 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb981-a6ad-4896-abdd-85fe71cb199c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:22.721072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-921bb981-a6ad-4896-abdd-85fe71cb199c None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:22.722273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-921bb981-a6ad-4896-abdd-85fe71cb199c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:22.731114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1344/2706] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:22.735187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3502229-1d1b-4061-9aeb-70b32d80f5ee None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:22.744870 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3502229-1d1b-4061-9aeb-70b32d80f5ee None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:22.749316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c62a7-e56b-46e0-bbb4-7570e8876ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:22.749899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c62a7-e56b-46e0-bbb4-7570e8876ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:22.750178 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c62a7-e56b-46e0-bbb4-7570e8876ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:22.750442 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c62a7-e56b-46e0-bbb4-7570e8876ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:22.751073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c62a7-e56b-46e0-bbb4-7570e8876ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:39:22.751367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c62a7-e56b-46e0-bbb4-7570e8876ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:22.752972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c62a7-e56b-46e0-bbb4-7570e8876ea3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:22.764507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1363/2707] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:39:22.768596 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bb981-a6ad-4896-abdd-85fe71cb199c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:22.778542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bb981-a6ad-4896-abdd-85fe71cb199c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:22.783231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c1d0e-7732-4156-9f62-7c4d47e47809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:22.783835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c1d0e-7732-4156-9f62-7c4d47e47809 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:22.784151 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c1d0e-7732-4156-9f62-7c4d47e47809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:22.784429 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c1d0e-7732-4156-9f62-7c4d47e47809 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:22.785849 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c1d0e-7732-4156-9f62-7c4d47e47809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:22.787749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c1d0e-7732-4156-9f62-7c4d47e47809 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:22.816212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1345/2708] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:39:22 2022] POST /identity/v3/projects => generated 397 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:39:22.820753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c62a7-e56b-46e0-bbb4-7570e8876ea3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:22.830729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c62a7-e56b-46e0-bbb4-7570e8876ea3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:22.848354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=100 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:22.886147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494724a-7926-478f-af77-00d488e1291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:22.886661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494724a-7926-478f-af77-00d488e1291b admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:22.886939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494724a-7926-478f-af77-00d488e1291b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:22.887209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494724a-7926-478f-af77-00d488e1291b admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:22.887818 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494724a-7926-478f-af77-00d488e1291b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:22.889752 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494724a-7926-478f-af77-00d488e1291b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:22.913013 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1364/2709] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:39:22 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:39:22.917254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c1d0e-7732-4156-9f62-7c4d47e47809 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:22.928566 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c1d0e-7732-4156-9f62-7c4d47e47809 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:22.933592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ef692-0cc7-434d-a7cb-bcf08dd05282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:22.934197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ef692-0cc7-434d-a7cb-bcf08dd05282 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:22.934479 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ef692-0cc7-434d-a7cb-bcf08dd05282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:22.934754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ef692-0cc7-434d-a7cb-bcf08dd05282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:22.935463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ef692-0cc7-434d-a7cb-bcf08dd05282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:22.937119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ef692-0cc7-434d-a7cb-bcf08dd05282 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:22.947303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1346/2710] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:39:22.951520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494724a-7926-478f-af77-00d488e1291b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:22.962538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494724a-7926-478f-af77-00d488e1291b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:22.967567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6582f3-9903-42fe-9af1-74c49ce25366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:22.968198 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6582f3-9903-42fe-9af1-74c49ce25366 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:22.968524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6582f3-9903-42fe-9af1-74c49ce25366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:22.968795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6582f3-9903-42fe-9af1-74c49ce25366 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:22.969430 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6582f3-9903-42fe-9af1-74c49ce25366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:22.971024 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6582f3-9903-42fe-9af1-74c49ce25366 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:22.982426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1365/2711] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:39:22.986576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ef692-0cc7-434d-a7cb-bcf08dd05282 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:22.997580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ef692-0cc7-434d-a7cb-bcf08dd05282 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:23.002247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930d27c-c5e4-4c04-87dd-d0487cb185ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:23.002875 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930d27c-c5e4-4c04-87dd-d0487cb185ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:23.003274 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930d27c-c5e4-4c04-87dd-d0487cb185ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:23.004066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930d27c-c5e4-4c04-87dd-d0487cb185ee admin admin] PATH_INFO: `/v3/projects/6651503fad2949538c3bad2d7a0aaada/users/6fdb1b788dee4d6cb9494db6f55b2a9f/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:23.019966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930d27c-c5e4-4c04-87dd-d0487cb185ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=6651503fad2949538c3bad2d7a0aaada, user_id=6fdb1b788dee4d6cb9494db6f55b2a9f, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:23.022654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930d27c-c5e4-4c04-87dd-d0487cb185ee admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:23.038794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1347/2712] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:39:22 2022] PUT /identity/v3/projects/6651503fad2949538c3bad2d7a0aaada/users/6fdb1b788dee4d6cb9494db6f55b2a9f/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:23.049090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf5e62-56a0-4916-b5da-559a8b8d7dde None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:23.049580 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf5e62-56a0-4916-b5da-559a8b8d7dde None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:23.049688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf5e62-56a0-4916-b5da-559a8b8d7dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:23.077026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-29bf5e62-56a0-4916-b5da-559a8b8d7dde None None] MFA Rules not processed for user `6fdb1b788dee4d6cb9494db6f55b2a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:23.078240 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-29bf5e62-56a0-4916-b5da-559a8b8d7dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:23.142133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1366/2713] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:23 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:23.154524 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930d27c-c5e4-4c04-87dd-d0487cb185ee admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:23.156300 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d930d27c-c5e4-4c04-87dd-d0487cb185ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:23.215369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:23.221475 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930d27c-c5e4-4c04-87dd-d0487cb185ee admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:23.226220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcfb53-954d-41b5-a3e8-64ee63468683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:23.226894 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcfb53-954d-41b5-a3e8-64ee63468683 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:23.227241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcfb53-954d-41b5-a3e8-64ee63468683 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:23.227578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcfb53-954d-41b5-a3e8-64ee63468683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:23.237516 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcfb53-954d-41b5-a3e8-64ee63468683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:23.238982 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcfb53-954d-41b5-a3e8-64ee63468683 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:23.260070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1348/2714] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:23 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:25.068825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf5e62-56a0-4916-b5da-559a8b8d7dde None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:25.070236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-29bf5e62-56a0-4916-b5da-559a8b8d7dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:25.134780 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf5e62-56a0-4916-b5da-559a8b8d7dde None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:25.139167 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:25.139790 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:25.140128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:25.140365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] PATH_INFO: `/v3/users/b434694b8bc3452fbbf43e65f7626666` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:25.141484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b434694b8bc3452fbbf43e65f7626666)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:25.143463 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:25.178174 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:25.193740 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b434694b8bc3452fbbf43e65f7626666'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:25.194397 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b434694b8bc3452fbbf43e65f7626666'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:25.195003 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b434694b8bc3452fbbf43e65f7626666'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:25.202054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b434694b8bc3452fbbf43e65f7626666'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:25.209463 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b434694b8bc3452fbbf43e65f7626666'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:25.227528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1367/2715] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:39:25 2022] DELETE /identity/v3/users/b434694b8bc3452fbbf43e65f7626666 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:25.231995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e00f69-4ad8-4b24-9b38-9856afd42a8a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:25.232539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e00f69-4ad8-4b24-9b38-9856afd42a8a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:25.232657 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e00f69-4ad8-4b24-9b38-9856afd42a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:25.250755 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-42e00f69-4ad8-4b24-9b38-9856afd42a8a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:25.251936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-42e00f69-4ad8-4b24-9b38-9856afd42a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:25.311986 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1349/2716] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:25.322181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:25.323575 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:25.382356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2e094-d2bd-479f-bb12-51507ef6af04 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:25.385940 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42331c-aadb-4094-b1e0-757c0f303f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:25.386538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42331c-aadb-4094-b1e0-757c0f303f57 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:25.386820 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42331c-aadb-4094-b1e0-757c0f303f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:25.387096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42331c-aadb-4094-b1e0-757c0f303f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:25.396727 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42331c-aadb-4094-b1e0-757c0f303f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:25.398215 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42331c-aadb-4094-b1e0-757c0f303f57 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:25.407516 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1368/2717] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:25.679152 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e00f69-4ad8-4b24-9b38-9856afd42a8a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:25.680722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-42e00f69-4ad8-4b24-9b38-9856afd42a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:25.694179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e00f69-4ad8-4b24-9b38-9856afd42a8a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:25.699394 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:25.700112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:25.700410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:25.700685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] PATH_INFO: `/v3/projects/d896e5cd66cb46b4863d11c9492bd72e` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:25.701856 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d896e5cd66cb46b4863d11c9492bd72e)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:25.704544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:25.754495 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd896e5cd66cb46b4863d11c9492bd72e', 'request_id': 'req-ede76ab5-2152-4fbc-a426-27ce30e8e10a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:25.760600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd896e5cd66cb46b4863d11c9492bd72e', 'request_id': 'req-ede76ab5-2152-4fbc-a426-27ce30e8e10a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:25.766258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd896e5cd66cb46b4863d11c9492bd72e', 'request_id': 'req-ede76ab5-2152-4fbc-a426-27ce30e8e10a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:25.779234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] The token cache is being invalidate because project d896e5cd66cb46b4863d11c9492bd72e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:39:25.779525 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:25.780186 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ede76ab5-2152-4fbc-a426-27ce30e8e10a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:25.781703 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1350/2718] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:39:25 2022] DELETE /identity/v3/projects/d896e5cd66cb46b4863d11c9492bd72e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:25.786880 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60b1e8-b114-4835-ba40-f43489d3edc9 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:25.787165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60b1e8-b114-4835-ba40-f43489d3edc9 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:25.787438 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60b1e8-b114-4835-ba40-f43489d3edc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:25.808007 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-8b60b1e8-b114-4835-ba40-f43489d3edc9 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:25.809224 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-8b60b1e8-b114-4835-ba40-f43489d3edc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:25.867052 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1369/2719] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:39:25.874804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873f99c-8cce-40fd-97ed-0f71f4319752 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:25.875096 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873f99c-8cce-40fd-97ed-0f71f4319752 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:25.875396 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873f99c-8cce-40fd-97ed-0f71f4319752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:25.895129 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6873f99c-8cce-40fd-97ed-0f71f4319752 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:25.896369 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6873f99c-8cce-40fd-97ed-0f71f4319752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:25.904108 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1351/2720] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:39:25.908219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60b1e8-b114-4835-ba40-f43489d3edc9 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:25.918137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60b1e8-b114-4835-ba40-f43489d3edc9 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:25.922494 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510d18c-9693-4b7d-a3da-1e5003b3afc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:25.923070 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510d18c-9693-4b7d-a3da-1e5003b3afc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:25.923344 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510d18c-9693-4b7d-a3da-1e5003b3afc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:25.923610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510d18c-9693-4b7d-a3da-1e5003b3afc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:25.924282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510d18c-9693-4b7d-a3da-1e5003b3afc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:39:25.924579 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510d18c-9693-4b7d-a3da-1e5003b3afc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:25.926656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510d18c-9693-4b7d-a3da-1e5003b3afc1 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:25.937932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1370/2721] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:39:25.942770 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873f99c-8cce-40fd-97ed-0f71f4319752 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:25.952380 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873f99c-8cce-40fd-97ed-0f71f4319752 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:25.957097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc72489-cedb-45e0-a9b2-36c58b9daaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:25.957698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc72489-cedb-45e0-a9b2-36c58b9daaa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:25.958014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc72489-cedb-45e0-a9b2-36c58b9daaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:25.958287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc72489-cedb-45e0-a9b2-36c58b9daaa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:25.958917 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc72489-cedb-45e0-a9b2-36c58b9daaa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:25.960862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc72489-cedb-45e0-a9b2-36c58b9daaa5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:25.989935 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1352/2722] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:39:25 2022] POST /identity/v3/projects => generated 359 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:39:25.994087 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510d18c-9693-4b7d-a3da-1e5003b3afc1 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:26.003947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510d18c-9693-4b7d-a3da-1e5003b3afc1 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:26.037366 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:26.062807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98088d74-679a-4d89-9cb0-608e3417b63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:26.063420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98088d74-679a-4d89-9cb0-608e3417b63b admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:26.063686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98088d74-679a-4d89-9cb0-608e3417b63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:26.063961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98088d74-679a-4d89-9cb0-608e3417b63b admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:26.064621 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98088d74-679a-4d89-9cb0-608e3417b63b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:26.066570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98088d74-679a-4d89-9cb0-608e3417b63b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:26.086961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1371/2723] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:39:25 2022] POST /identity/v3/users => generated 280 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:39:26.091198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc72489-cedb-45e0-a9b2-36c58b9daaa5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:26.101795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc72489-cedb-45e0-a9b2-36c58b9daaa5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:26.106489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a88c4-611d-4a2c-9a55-8851151a6b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:26.107074 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a88c4-611d-4a2c-9a55-8851151a6b66 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:26.107344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a88c4-611d-4a2c-9a55-8851151a6b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:26.107614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a88c4-611d-4a2c-9a55-8851151a6b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:26.108283 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a88c4-611d-4a2c-9a55-8851151a6b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:26.109942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a88c4-611d-4a2c-9a55-8851151a6b66 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:26.120651 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1353/2724] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:39:26.124781 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98088d74-679a-4d89-9cb0-608e3417b63b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:26.134835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98088d74-679a-4d89-9cb0-608e3417b63b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:26.139447 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012496f-6f3e-4504-885b-cfe8ebff0aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:26.140036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012496f-6f3e-4504-885b-cfe8ebff0aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:26.140345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012496f-6f3e-4504-885b-cfe8ebff0aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:26.140619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012496f-6f3e-4504-885b-cfe8ebff0aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:26.141259 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012496f-6f3e-4504-885b-cfe8ebff0aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:26.142867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012496f-6f3e-4504-885b-cfe8ebff0aa9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:26.153838 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1372/2725] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:39:26.158072 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a88c4-611d-4a2c-9a55-8851151a6b66 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:26.168993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a88c4-611d-4a2c-9a55-8851151a6b66 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:26.173804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a5d65-7495-41c4-b864-9519fa4638e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:26.174452 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a5d65-7495-41c4-b864-9519fa4638e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:26.174736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a5d65-7495-41c4-b864-9519fa4638e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:26.175017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a5d65-7495-41c4-b864-9519fa4638e2 admin admin] PATH_INFO: `/v3/projects/c317b80c153946748e9bcbc4f70a5d3f/users/4efa5bf436884479b0a84d43dd1feadb/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:26.191527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a5d65-7495-41c4-b864-9519fa4638e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c317b80c153946748e9bcbc4f70a5d3f, user_id=4efa5bf436884479b0a84d43dd1feadb, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:26.194505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a5d65-7495-41c4-b864-9519fa4638e2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:26.208199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1354/2726] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:39:26 2022] PUT /identity/v3/projects/c317b80c153946748e9bcbc4f70a5d3f/users/4efa5bf436884479b0a84d43dd1feadb/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:26.219628 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae84fb-5054-43f8-b4d3-418105386691 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:26.219914 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae84fb-5054-43f8-b4d3-418105386691 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:26.220229 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae84fb-5054-43f8-b4d3-418105386691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:26.248473 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-efae84fb-5054-43f8-b4d3-418105386691 None None] MFA Rules not processed for user `4efa5bf436884479b0a84d43dd1feadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:26.249686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-efae84fb-5054-43f8-b4d3-418105386691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:26.312680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1373/2727] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:26.344826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a5d65-7495-41c4-b864-9519fa4638e2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:26.346631 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-464a5d65-7495-41c4-b864-9519fa4638e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:26.362334 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:26.409098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a5d65-7495-41c4-b864-9519fa4638e2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:26.413613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3f448-802b-4be9-840e-9342fb14c12c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:26.414210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3f448-802b-4be9-840e-9342fb14c12c service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:26.414487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3f448-802b-4be9-840e-9342fb14c12c service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:26.414756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3f448-802b-4be9-840e-9342fb14c12c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:26.422913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3f448-802b-4be9-840e-9342fb14c12c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:26.424311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3f448-802b-4be9-840e-9342fb14c12c service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:26.433819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1355/2728] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:26.450867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae84fb-5054-43f8-b4d3-418105386691 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:26.512015 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae84fb-5054-43f8-b4d3-418105386691 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:26.516521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:26.517130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:26.517406 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:26.517675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] PATH_INFO: `/v3/users/4efa5bf436884479b0a84d43dd1feadb` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:26.518791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4efa5bf436884479b0a84d43dd1feadb)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:26.521248 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:26.566390 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4efa5bf436884479b0a84d43dd1feadb'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:26.567034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4efa5bf436884479b0a84d43dd1feadb'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:26.567648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4efa5bf436884479b0a84d43dd1feadb'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:26.569307 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:26.576629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4efa5bf436884479b0a84d43dd1feadb'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:26.584904 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4efa5bf436884479b0a84d43dd1feadb'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:26.602645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1374/2729] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:39:26 2022] DELETE /identity/v3/users/4efa5bf436884479b0a84d43dd1feadb => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:26.607190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cd32c-0b39-4fc1-aa9d-a528ad944224 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:26.607474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cd32c-0b39-4fc1-aa9d-a528ad944224 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:26.607747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cd32c-0b39-4fc1-aa9d-a528ad944224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:26.626181 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6c7cd32c-0b39-4fc1-aa9d-a528ad944224 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:26.627375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6c7cd32c-0b39-4fc1-aa9d-a528ad944224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:26.684932 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1356/2730] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:26.695118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:26.696594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:26.753465 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee404fe5-3b9e-4659-a43a-1ec6330f8134 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:26.757577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2c338-1246-4980-b72d-1db3df42fdfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:26.758176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2c338-1246-4980-b72d-1db3df42fdfb service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:26.758452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2c338-1246-4980-b72d-1db3df42fdfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:26.758720 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2c338-1246-4980-b72d-1db3df42fdfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:26.768339 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2c338-1246-4980-b72d-1db3df42fdfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:26.769942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2c338-1246-4980-b72d-1db3df42fdfb service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:26.781558 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1375/2731] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:27.049695 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cd32c-0b39-4fc1-aa9d-a528ad944224 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:27.051294 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6c7cd32c-0b39-4fc1-aa9d-a528ad944224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:27.063867 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cd32c-0b39-4fc1-aa9d-a528ad944224 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:27.069201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:27.069826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:27.070109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:27.070383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] PATH_INFO: `/v3/projects/c317b80c153946748e9bcbc4f70a5d3f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:27.071538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c317b80c153946748e9bcbc4f70a5d3f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:27.073513 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:27.121474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c317b80c153946748e9bcbc4f70a5d3f', 'request_id': 'req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:27.127602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c317b80c153946748e9bcbc4f70a5d3f', 'request_id': 'req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:27.133744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c317b80c153946748e9bcbc4f70a5d3f', 'request_id': 'req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:27.145766 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] The token cache is being invalidate because project c317b80c153946748e9bcbc4f70a5d3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:39:27.146054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:27.146671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e2d0b407-ade2-42b8-a81d-26e6d4bd4f1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:27.148182 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1357/2732] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:39:27 2022] DELETE /identity/v3/projects/c317b80c153946748e9bcbc4f70a5d3f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:31.359865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2c338-1246-4980-b72d-1db3df42fdfb service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:31.361420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-63d2c338-1246-4980-b72d-1db3df42fdfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:31.373910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2c338-1246-4980-b72d-1db3df42fdfb service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:31.439020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:31.439986 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:31.440567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:31.440894 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] PATH_INFO: `/v3/users/6fdb1b788dee4d6cb9494db6f55b2a9f` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:31.442250 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fdb1b788dee4d6cb9494db6f55b2a9f)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:31.446337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:31.500829 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fdb1b788dee4d6cb9494db6f55b2a9f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:31.501530 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fdb1b788dee4d6cb9494db6f55b2a9f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:31.502131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fdb1b788dee4d6cb9494db6f55b2a9f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:31.510958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fdb1b788dee4d6cb9494db6f55b2a9f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:31.519486 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fdb1b788dee4d6cb9494db6f55b2a9f'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:31.525875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:31.550658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1376/2733] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:39:31 2022] DELETE /identity/v3/users/6fdb1b788dee4d6cb9494db6f55b2a9f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:31.555534 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b616ad7-c767-417f-bf44-36625cfd7bae None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:31.556006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b616ad7-c767-417f-bf44-36625cfd7bae None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:31.556142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b616ad7-c767-417f-bf44-36625cfd7bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:31.575712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7b616ad7-c767-417f-bf44-36625cfd7bae None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:31.576906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7b616ad7-c767-417f-bf44-36625cfd7bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:31.610418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:31.642293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1358/2734] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:31.651958 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:31.653379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:31.709219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f170c-e7c8-4112-8f72-4fa91c53a83f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:31.712930 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7a9eb-a374-400f-92c5-cbadf160ba5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:31.713520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7a9eb-a374-400f-92c5-cbadf160ba5b service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:31.713794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7a9eb-a374-400f-92c5-cbadf160ba5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:31.714060 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7a9eb-a374-400f-92c5-cbadf160ba5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:31.722794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7a9eb-a374-400f-92c5-cbadf160ba5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:31.724246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7a9eb-a374-400f-92c5-cbadf160ba5b service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:31.733624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1377/2735] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:32.019470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b616ad7-c767-417f-bf44-36625cfd7bae None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:32.021302 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7b616ad7-c767-417f-bf44-36625cfd7bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:32.033837 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b616ad7-c767-417f-bf44-36625cfd7bae None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:32.039838 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:32.040538 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:32.040984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:32.041340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] PATH_INFO: `/v3/projects/6651503fad2949538c3bad2d7a0aaada` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:32.042536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6651503fad2949538c3bad2d7a0aaada)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:32.044533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:32.095463 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6651503fad2949538c3bad2d7a0aaada', 'request_id': 'req-550df6ae-c474-4d15-bf8d-7438f46210e0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:32.102121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6651503fad2949538c3bad2d7a0aaada', 'request_id': 'req-550df6ae-c474-4d15-bf8d-7438f46210e0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:32.108107 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6651503fad2949538c3bad2d7a0aaada', 'request_id': 'req-550df6ae-c474-4d15-bf8d-7438f46210e0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:32.122304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] The token cache is being invalidate because project 6651503fad2949538c3bad2d7a0aaada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:39:32.122600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:32.123252 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:32.124801 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1359/2736] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:39:32 2022] DELETE /identity/v3/projects/6651503fad2949538c3bad2d7a0aaada => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:41.729559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=31,keystone:DELETE=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:42.118773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:43.221696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7a9eb-a374-400f-92c5-cbadf160ba5b service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:43.223085 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d7a7a9eb-a374-400f-92c5-cbadf160ba5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:43.247078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7a9eb-a374-400f-92c5-cbadf160ba5b service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:43.295306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51955986-a332-44fb-bbcb-4197ce070f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:43.295898 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51955986-a332-44fb-bbcb-4197ce070f71 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:43.296207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51955986-a332-44fb-bbcb-4197ce070f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:43.296474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51955986-a332-44fb-bbcb-4197ce070f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:43.298187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-51955986-a332-44fb-bbcb-4197ce070f71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:43.360000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51955986-a332-44fb-bbcb-4197ce070f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:43.361515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51955986-a332-44fb-bbcb-4197ce070f71 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:43.371872 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1378/2737] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:46.498474 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:46.500206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:46.514024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550df6ae-c474-4d15-bf8d-7438f46210e0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:46.518627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c7337-13a0-4905-9719-ff99b15f06c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:46.519252 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c7337-13a0-4905-9719-ff99b15f06c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:46.519533 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c7337-13a0-4905-9719-ff99b15f06c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:46.519813 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c7337-13a0-4905-9719-ff99b15f06c2 admin admin] PATH_INFO: `/v3/users/1de97cce55e944dbb62a920ea11734f5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:46.520998 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c7337-13a0-4905-9719-ff99b15f06c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de97cce55e944dbb62a920ea11734f5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:46.522925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c7337-13a0-4905-9719-ff99b15f06c2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:46.571500 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-951c7337-13a0-4905-9719-ff99b15f06c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de97cce55e944dbb62a920ea11734f5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:46.572730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-951c7337-13a0-4905-9719-ff99b15f06c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de97cce55e944dbb62a920ea11734f5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:46.573347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-951c7337-13a0-4905-9719-ff99b15f06c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de97cce55e944dbb62a920ea11734f5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:46.580931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-951c7337-13a0-4905-9719-ff99b15f06c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de97cce55e944dbb62a920ea11734f5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:46.588971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-951c7337-13a0-4905-9719-ff99b15f06c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de97cce55e944dbb62a920ea11734f5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:46.606305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1360/2738] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:39:46 2022] DELETE /identity/v3/users/1de97cce55e944dbb62a920ea11734f5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:46.779234 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51955986-a332-44fb-bbcb-4197ce070f71 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:46.790197 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51955986-a332-44fb-bbcb-4197ce070f71 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:46.847493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:46.848148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:46.848444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:46.848752 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] PATH_INFO: `/v3/projects/e055ba4967db42108c069ebc3843dcf6` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:46.849919 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e055ba4967db42108c069ebc3843dcf6)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:46.851833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:46.891771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:46.899386 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e055ba4967db42108c069ebc3843dcf6', 'request_id': 'req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:46.905722 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e055ba4967db42108c069ebc3843dcf6', 'request_id': 'req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:46.911054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e055ba4967db42108c069ebc3843dcf6', 'request_id': 'req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:46.923847 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] The token cache is being invalidate because project e055ba4967db42108c069ebc3843dcf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:39:46.924171 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:46.924785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9aa2c8d8-1b3a-41ba-8651-21acc8c1a4cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:39:46.926315 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1379/2739] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:39:46 2022] DELETE /identity/v3/projects/e055ba4967db42108c069ebc3843dcf6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:46.931308 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf3423-3fbb-4d4c-8614-9babd888d752 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:46.932216 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf3423-3fbb-4d4c-8614-9babd888d752 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:46.932492 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf3423-3fbb-4d4c-8614-9babd888d752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:46.952610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4abf3423-3fbb-4d4c-8614-9babd888d752 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:46.953809 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4abf3423-3fbb-4d4c-8614-9babd888d752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:47.012088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1361/2740] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:47.019993 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1835d3-0631-4591-b967-d1003153376a None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:47.020323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1835d3-0631-4591-b967-d1003153376a None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:47.020600 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1835d3-0631-4591-b967-d1003153376a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:47.041732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ca1835d3-0631-4591-b967-d1003153376a None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:47.042939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ca1835d3-0631-4591-b967-d1003153376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:47.050865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1380/2741] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:47.055038 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf3423-3fbb-4d4c-8614-9babd888d752 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:47.065264 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf3423-3fbb-4d4c-8614-9babd888d752 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:47.069858 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c542f9c-f956-42fa-a502-fa6fd98afd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:47.070446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c542f9c-f956-42fa-a502-fa6fd98afd35 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:47.070726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c542f9c-f956-42fa-a502-fa6fd98afd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:47.070999 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c542f9c-f956-42fa-a502-fa6fd98afd35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:47.071642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c542f9c-f956-42fa-a502-fa6fd98afd35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:39:47.071968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c542f9c-f956-42fa-a502-fa6fd98afd35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:47.073597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c542f9c-f956-42fa-a502-fa6fd98afd35 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:47.084957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1362/2742] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:39:47.089196 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1835d3-0631-4591-b967-d1003153376a None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:47.098665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1835d3-0631-4591-b967-d1003153376a None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:47.103758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9fc72-0ed1-458f-99a0-62d9adcace33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:47.104375 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9fc72-0ed1-458f-99a0-62d9adcace33 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:47.104713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9fc72-0ed1-458f-99a0-62d9adcace33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:47.104916 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9fc72-0ed1-458f-99a0-62d9adcace33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:47.105526 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9fc72-0ed1-458f-99a0-62d9adcace33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:47.107407 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9fc72-0ed1-458f-99a0-62d9adcace33 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:47.134269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1381/2743] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:39:47 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:39:47.138618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c542f9c-f956-42fa-a502-fa6fd98afd35 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:47.148599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c542f9c-f956-42fa-a502-fa6fd98afd35 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:47.205029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f7873-7916-4ddc-961d-d9c6d84b4593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:47.205633 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f7873-7916-4ddc-961d-d9c6d84b4593 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:47.205916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f7873-7916-4ddc-961d-d9c6d84b4593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:47.206196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f7873-7916-4ddc-961d-d9c6d84b4593 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:47.206823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f7873-7916-4ddc-961d-d9c6d84b4593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:47.208826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f7873-7916-4ddc-961d-d9c6d84b4593 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:47.216849 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:47.236689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1363/2744] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:39:47 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:39:47.237869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9fc72-0ed1-458f-99a0-62d9adcace33 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:47.249016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9fc72-0ed1-458f-99a0-62d9adcace33 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:47.253920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fef327-3002-4040-bb78-87e4d79200c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:47.254523 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef327-3002-4040-bb78-87e4d79200c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:47.254805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef327-3002-4040-bb78-87e4d79200c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:47.255080 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef327-3002-4040-bb78-87e4d79200c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:47.255723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fef327-3002-4040-bb78-87e4d79200c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:47.257359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fef327-3002-4040-bb78-87e4d79200c0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:47.268101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1382/2745] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:39:47.271956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f7873-7916-4ddc-961d-d9c6d84b4593 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:47.282442 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f7873-7916-4ddc-961d-d9c6d84b4593 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:47.287049 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a45ab-aeed-4c26-9052-008c6884f6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:47.287906 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a45ab-aeed-4c26-9052-008c6884f6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:47.288222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a45ab-aeed-4c26-9052-008c6884f6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:47.288497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a45ab-aeed-4c26-9052-008c6884f6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:47.289139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a45ab-aeed-4c26-9052-008c6884f6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:47.291342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a45ab-aeed-4c26-9052-008c6884f6b7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:47.301925 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1364/2746] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:39:47.305843 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fef327-3002-4040-bb78-87e4d79200c0 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:47.315645 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fef327-3002-4040-bb78-87e4d79200c0 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:47.320161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0416eff-44fc-4936-951b-14f8e7efe881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:47.320789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416eff-44fc-4936-951b-14f8e7efe881 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:47.321067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416eff-44fc-4936-951b-14f8e7efe881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:47.321338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416eff-44fc-4936-951b-14f8e7efe881 admin admin] PATH_INFO: `/v3/projects/9569b373f21841e8883b0b269073121a/users/311b0a8b22174a278793537963b5d142/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:47.335805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0416eff-44fc-4936-951b-14f8e7efe881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9569b373f21841e8883b0b269073121a, user_id=311b0a8b22174a278793537963b5d142, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:47.338992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0416eff-44fc-4936-951b-14f8e7efe881 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:47.353973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1383/2747] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:39:47 2022] PUT /identity/v3/projects/9569b373f21841e8883b0b269073121a/users/311b0a8b22174a278793537963b5d142/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:39:47.359092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b5104-7c5d-4e76-8791-5fd5377fb43e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:47.359385 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b5104-7c5d-4e76-8791-5fd5377fb43e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:47.359689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b5104-7c5d-4e76-8791-5fd5377fb43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:47.378239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ab7b5104-7c5d-4e76-8791-5fd5377fb43e None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:47.379485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab7b5104-7c5d-4e76-8791-5fd5377fb43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:47.437935 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1365/2748] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:47.447589 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0416eff-44fc-4936-951b-14f8e7efe881 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:47.449363 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e0416eff-44fc-4936-951b-14f8e7efe881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:47.506047 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0416eff-44fc-4936-951b-14f8e7efe881 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:47.509508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ca0ed-ea9b-41fd-b911-d3f3a1303c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:47.510098 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ca0ed-ea9b-41fd-b911-d3f3a1303c02 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:47.510393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ca0ed-ea9b-41fd-b911-d3f3a1303c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:47.510644 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ca0ed-ea9b-41fd-b911-d3f3a1303c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:47.519428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ca0ed-ea9b-41fd-b911-d3f3a1303c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:47.520888 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ca0ed-ea9b-41fd-b911-d3f3a1303c02 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:47.530531 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1384/2749] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:51.202051 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf6152-4e9c-4108-aba6-c878cf3a86c9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:51.202578 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf6152-4e9c-4108-aba6-c878cf3a86c9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:51.202709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf6152-4e9c-4108-aba6-c878cf3a86c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:51.231054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bcaf6152-4e9c-4108-aba6-c878cf3a86c9 None None] MFA Rules not processed for user `311b0a8b22174a278793537963b5d142`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:51.232850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bcaf6152-4e9c-4108-aba6-c878cf3a86c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:51.301487 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1366/2750] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:51.318053 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ca0ed-ea9b-41fd-b911-d3f3a1303c02 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:51.319585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ca9ca0ed-ea9b-41fd-b911-d3f3a1303c02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:51.393750 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ca0ed-ea9b-41fd-b911-d3f3a1303c02 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:51.399073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185bc2-f5a3-4155-b83b-25d87fbe8927 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:51.399682 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185bc2-f5a3-4155-b83b-25d87fbe8927 service nova] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:51.399947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185bc2-f5a3-4155-b83b-25d87fbe8927 service nova] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:51.400281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185bc2-f5a3-4155-b83b-25d87fbe8927 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:51.406324 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:51.409901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5185bc2-f5a3-4155-b83b-25d87fbe8927 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:51.411552 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5185bc2-f5a3-4155-b83b-25d87fbe8927 service nova] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:51.423227 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1385/2751] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:39:52.791606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cb1f3-a4f5-4fc4-b3f2-b806b010de66 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:52.792160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cb1f3-a4f5-4fc4-b3f2-b806b010de66 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:52.792292 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cb1f3-a4f5-4fc4-b3f2-b806b010de66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:52.811485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ba4cb1f3-a4f5-4fc4-b3f2-b806b010de66 None None] MFA Rules not processed for user `311b0a8b22174a278793537963b5d142`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:39:52.812710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ba4cb1f3-a4f5-4fc4-b3f2-b806b010de66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:52.819729 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1367/2752] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:39:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:39:53.454679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185bc2-f5a3-4155-b83b-25d87fbe8927 service nova] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:39:53.456839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-b5185bc2-f5a3-4155-b83b-25d87fbe8927 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:53.536764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185bc2-f5a3-4155-b83b-25d87fbe8927 service nova] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:39:53.542684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320f6e8-a563-47c7-8dd1-b5b1a2a58f89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:39:53.543393 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320f6e8-a563-47c7-8dd1-b5b1a2a58f89 service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:39:53.543673 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320f6e8-a563-47c7-8dd1-b5b1a2a58f89 service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:39:53.543941 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320f6e8-a563-47c7-8dd1-b5b1a2a58f89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:39:53.545690 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3320f6e8-a563-47c7-8dd1-b5b1a2a58f89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:39:53.557418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320f6e8-a563-47c7-8dd1-b5b1a2a58f89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:39:53.559067 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320f6e8-a563-47c7-8dd1-b5b1a2a58f89 service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:39:53.569701 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1386/2753] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:39:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:40:02.693335 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cb1f3-a4f5-4fc4-b3f2-b806b010de66 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:40:02.694792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ba4cb1f3-a4f5-4fc4-b3f2-b806b010de66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:40:02.711489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cb1f3-a4f5-4fc4-b3f2-b806b010de66 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:40:02.717278 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f530d2e-8810-4431-9e47-7cfb8b49d6f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:40:02.717878 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f530d2e-8810-4431-9e47-7cfb8b49d6f6 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:40:02.718160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f530d2e-8810-4431-9e47-7cfb8b49d6f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:40:02.718434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f530d2e-8810-4431-9e47-7cfb8b49d6f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:40:02.720222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5f530d2e-8810-4431-9e47-7cfb8b49d6f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:40:02.733077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f530d2e-8810-4431-9e47-7cfb8b49d6f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:40:02.734647 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f530d2e-8810-4431-9e47-7cfb8b49d6f6 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:40:02.746121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1368/2754] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:40:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:40:03.565061 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=35 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:05.380332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320f6e8-a563-47c7-8dd1-b5b1a2a58f89 service placement] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:40:05.390496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320f6e8-a563-47c7-8dd1-b5b1a2a58f89 service placement] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:40:05.394105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d809d3-cc3d-4e7c-be40-3964b0f4aaf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:40:05.394711 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d809d3-cc3d-4e7c-be40-3964b0f4aaf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:40:05.394992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d809d3-cc3d-4e7c-be40-3964b0f4aaf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:40:05.395266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d809d3-cc3d-4e7c-be40-3964b0f4aaf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:40:05.397210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-52d809d3-cc3d-4e7c-be40-3964b0f4aaf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:40:05.411687 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d809d3-cc3d-4e7c-be40-3964b0f4aaf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:40:05.413150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d809d3-cc3d-4e7c-be40-3964b0f4aaf9 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:40:05.424529 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1387/2755] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:40:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:40:12.740724 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=3,keystone:SELECT=75 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:15.419659 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:42.652274 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f530d2e-8810-4431-9e47-7cfb8b49d6f6 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:40:42.661502 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f530d2e-8810-4431-9e47-7cfb8b49d6f6 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:40:42.665077 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac97e0e-5ece-45f7-a6df-6bd3ba6688a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:40:42.665666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac97e0e-5ece-45f7-a6df-6bd3ba6688a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:40:42.666189 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac97e0e-5ece-45f7-a6df-6bd3ba6688a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:40:42.666489 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac97e0e-5ece-45f7-a6df-6bd3ba6688a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:40:42.668190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eac97e0e-5ece-45f7-a6df-6bd3ba6688a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:40:42.679256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac97e0e-5ece-45f7-a6df-6bd3ba6688a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:40:42.680721 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac97e0e-5ece-45f7-a6df-6bd3ba6688a5 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:40:42.689416 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1369/2756] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:40:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:40:52.685748 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:53.381706 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb423c-e24c-40ea-9d3b-3c21ab4d6e97 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:40:53.382207 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb423c-e24c-40ea-9d3b-3c21ab4d6e97 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:40:53.382301 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb423c-e24c-40ea-9d3b-3c21ab4d6e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:40:53.400883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1dcb423c-e24c-40ea-9d3b-3c21ab4d6e97 None None] MFA Rules not processed for user `311b0a8b22174a278793537963b5d142`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:40:53.402286 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1dcb423c-e24c-40ea-9d3b-3c21ab4d6e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:40:53.408922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1388/2757] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:40:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:40:53.714674 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac97e0e-5ece-45f7-a6df-6bd3ba6688a5 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:40:53.723757 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac97e0e-5ece-45f7-a6df-6bd3ba6688a5 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:40:53.727425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f17c09-a5e0-42af-a498-fee4de5d8fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:40:53.728014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f17c09-a5e0-42af-a498-fee4de5d8fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:40:53.728320 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f17c09-a5e0-42af-a498-fee4de5d8fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:40:53.728594 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f17c09-a5e0-42af-a498-fee4de5d8fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:40:53.730560 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-26f17c09-a5e0-42af-a498-fee4de5d8fa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:40:53.744581 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f17c09-a5e0-42af-a498-fee4de5d8fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:40:53.746023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f17c09-a5e0-42af-a498-fee4de5d8fa0 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:40:53.754739 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1370/2758] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:40:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:40:54.074334 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb423c-e24c-40ea-9d3b-3c21ab4d6e97 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:40:54.076199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1dcb423c-e24c-40ea-9d3b-3c21ab4d6e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:40:54.088008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb423c-e24c-40ea-9d3b-3c21ab4d6e97 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:40:54.092723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de44610-7bbe-4545-aecf-78bfefb625af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:40:54.093310 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de44610-7bbe-4545-aecf-78bfefb625af service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:40:54.093585 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de44610-7bbe-4545-aecf-78bfefb625af service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:40:54.093877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de44610-7bbe-4545-aecf-78bfefb625af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:40:54.095559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1de44610-7bbe-4545-aecf-78bfefb625af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:40:54.121463 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de44610-7bbe-4545-aecf-78bfefb625af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:40:54.122908 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de44610-7bbe-4545-aecf-78bfefb625af service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:40:54.133942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1389/2759] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:40:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:41:03.750916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:41:04.129030 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=14 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:41:58.573535 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892af2cd-11e7-4400-b49a-fae60e11c4a8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:41:58.574009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892af2cd-11e7-4400-b49a-fae60e11c4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:41:58.574098 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892af2cd-11e7-4400-b49a-fae60e11c4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:41:58.592852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-892af2cd-11e7-4400-b49a-fae60e11c4a8 None None] MFA Rules not processed for user `311b0a8b22174a278793537963b5d142`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:41:58.594018 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-892af2cd-11e7-4400-b49a-fae60e11c4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:41:58.600835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1371/2760] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:41:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:42:08.419716 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de44610-7bbe-4545-aecf-78bfefb625af service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:08.437789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de44610-7bbe-4545-aecf-78bfefb625af service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:08.441456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c99e3c-3c9f-4a34-8b75-278e9347257f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:08.442040 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c99e3c-3c9f-4a34-8b75-278e9347257f service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:08.442323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c99e3c-3c9f-4a34-8b75-278e9347257f service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:08.442601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c99e3c-3c9f-4a34-8b75-278e9347257f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:08.444346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-39c99e3c-3c9f-4a34-8b75-278e9347257f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:08.505186 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c99e3c-3c9f-4a34-8b75-278e9347257f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:08.506814 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c99e3c-3c9f-4a34-8b75-278e9347257f service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:08.526354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1390/2761] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:42:08.588009 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:12.072024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892af2cd-11e7-4400-b49a-fae60e11c4a8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:12.073508 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-892af2cd-11e7-4400-b49a-fae60e11c4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:12.121913 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892af2cd-11e7-4400-b49a-fae60e11c4a8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:12.126602 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c950a-3f8f-42c3-88b5-f254c149d109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:12.127220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c950a-3f8f-42c3-88b5-f254c149d109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:12.127663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c950a-3f8f-42c3-88b5-f254c149d109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:12.128128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c950a-3f8f-42c3-88b5-f254c149d109 admin admin] PATH_INFO: `/v3/users/df9b402589fa4020bc241a4b9c06dd91` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:12.129372 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c950a-3f8f-42c3-88b5-f254c149d109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df9b402589fa4020bc241a4b9c06dd91)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:12.131304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c950a-3f8f-42c3-88b5-f254c149d109 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:12.189966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-001c950a-3f8f-42c3-88b5-f254c149d109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df9b402589fa4020bc241a4b9c06dd91'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:12.191025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-001c950a-3f8f-42c3-88b5-f254c149d109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df9b402589fa4020bc241a4b9c06dd91'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:12.191880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-001c950a-3f8f-42c3-88b5-f254c149d109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df9b402589fa4020bc241a4b9c06dd91'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:12.201734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-001c950a-3f8f-42c3-88b5-f254c149d109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df9b402589fa4020bc241a4b9c06dd91'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:12.208990 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-001c950a-3f8f-42c3-88b5-f254c149d109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df9b402589fa4020bc241a4b9c06dd91'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:12.227053 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1372/2762] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:42:12 2022] DELETE /identity/v3/users/df9b402589fa4020bc241a4b9c06dd91 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:12.401457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c99e3c-3c9f-4a34-8b75-278e9347257f service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:12.412607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c99e3c-3c9f-4a34-8b75-278e9347257f service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:12.507542 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:12.508223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:12.508502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:12.508772 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] PATH_INFO: `/v3/projects/7c1ad7a7198d4e78b9d03f00ce65e091` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:12.509925 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c1ad7a7198d4e78b9d03f00ce65e091)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:12.511980 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:12.567305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c1ad7a7198d4e78b9d03f00ce65e091', 'request_id': 'req-4bbd4960-90e5-47ff-8222-0af758bcd4a6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:12.580877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c1ad7a7198d4e78b9d03f00ce65e091', 'request_id': 'req-4bbd4960-90e5-47ff-8222-0af758bcd4a6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:12.586679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c1ad7a7198d4e78b9d03f00ce65e091', 'request_id': 'req-4bbd4960-90e5-47ff-8222-0af758bcd4a6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:12.600634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] The token cache is being invalidate because project 7c1ad7a7198d4e78b9d03f00ce65e091 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:42:12.600929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:12.601563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-4bbd4960-90e5-47ff-8222-0af758bcd4a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:12.603054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1391/2763] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:42:12 2022] DELETE /identity/v3/projects/7c1ad7a7198d4e78b9d03f00ce65e091 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:12.609515 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73542ab0-9423-4ff1-91a5-af3b255f7f11 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:12.609833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73542ab0-9423-4ff1-91a5-af3b255f7f11 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:12.610109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73542ab0-9423-4ff1-91a5-af3b255f7f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:12.629671 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-73542ab0-9423-4ff1-91a5-af3b255f7f11 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:42:12.630957 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-73542ab0-9423-4ff1-91a5-af3b255f7f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:12.693744 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1373/2764] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:42:12.701910 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cde4c-c802-4ac0-8a2b-273438292b53 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:12.702216 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cde4c-c802-4ac0-8a2b-273438292b53 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:12.702490 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cde4c-c802-4ac0-8a2b-273438292b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:12.723700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a79cde4c-c802-4ac0-8a2b-273438292b53 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:42:12.724982 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a79cde4c-c802-4ac0-8a2b-273438292b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:12.733984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1392/2765] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:42:12.738374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73542ab0-9423-4ff1-91a5-af3b255f7f11 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:12.749289 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73542ab0-9423-4ff1-91a5-af3b255f7f11 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:12.754034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b673206-4e5e-4408-a72d-099681bb462f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:12.754663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b673206-4e5e-4408-a72d-099681bb462f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:12.754942 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b673206-4e5e-4408-a72d-099681bb462f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:12.755211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b673206-4e5e-4408-a72d-099681bb462f admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:12.755855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b673206-4e5e-4408-a72d-099681bb462f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:42:12.756208 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b673206-4e5e-4408-a72d-099681bb462f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:12.757789 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b673206-4e5e-4408-a72d-099681bb462f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:12.771056 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1374/2766] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:42:12.774630 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cde4c-c802-4ac0-8a2b-273438292b53 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:12.784928 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cde4c-c802-4ac0-8a2b-273438292b53 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:12.789638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815ee75-582f-4656-8b50-fadb118679b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:12.790247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815ee75-582f-4656-8b50-fadb118679b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:12.790502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815ee75-582f-4656-8b50-fadb118679b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:12.790771 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815ee75-582f-4656-8b50-fadb118679b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:12.791387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815ee75-582f-4656-8b50-fadb118679b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:12.793306 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815ee75-582f-4656-8b50-fadb118679b7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:12.804423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:12.824624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1393/2767] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:42:12 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:42:12.829450 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b673206-4e5e-4408-a72d-099681bb462f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:12.839567 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b673206-4e5e-4408-a72d-099681bb462f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:12.870094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:12.898094 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b290376-8daa-4c58-8c54-3a92ba644ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:12.898691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b290376-8daa-4c58-8c54-3a92ba644ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:12.898967 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b290376-8daa-4c58-8c54-3a92ba644ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:12.899236 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b290376-8daa-4c58-8c54-3a92ba644ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:12.899855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b290376-8daa-4c58-8c54-3a92ba644ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:12.901785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b290376-8daa-4c58-8c54-3a92ba644ff4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:12.922496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1375/2768] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:42:12 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:42:12.926671 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815ee75-582f-4656-8b50-fadb118679b7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:12.937432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815ee75-582f-4656-8b50-fadb118679b7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:12.941940 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b670fc2-3f1d-44d7-8de7-e760fe54a0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:12.942525 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b670fc2-3f1d-44d7-8de7-e760fe54a0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:12.942802 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b670fc2-3f1d-44d7-8de7-e760fe54a0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:12.943072 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b670fc2-3f1d-44d7-8de7-e760fe54a0f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:12.943736 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b670fc2-3f1d-44d7-8de7-e760fe54a0f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:12.945419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b670fc2-3f1d-44d7-8de7-e760fe54a0f7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:12.955631 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1394/2769] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:42:12.959371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b290376-8daa-4c58-8c54-3a92ba644ff4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:12.970332 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b290376-8daa-4c58-8c54-3a92ba644ff4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:12.974946 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a986a16-46bf-41ed-bbb2-9bdf284e0a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:12.975536 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a986a16-46bf-41ed-bbb2-9bdf284e0a4f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:12.975816 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a986a16-46bf-41ed-bbb2-9bdf284e0a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:12.976120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a986a16-46bf-41ed-bbb2-9bdf284e0a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:12.976760 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a986a16-46bf-41ed-bbb2-9bdf284e0a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:12.978342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a986a16-46bf-41ed-bbb2-9bdf284e0a4f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:12.988200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1376/2770] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:42:12.992219 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b670fc2-3f1d-44d7-8de7-e760fe54a0f7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:13.002374 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b670fc2-3f1d-44d7-8de7-e760fe54a0f7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:13.006912 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcac162-4e8c-4fe5-b1d4-98971601856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:13.007550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcac162-4e8c-4fe5-b1d4-98971601856d admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:13.007832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcac162-4e8c-4fe5-b1d4-98971601856d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:13.008138 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcac162-4e8c-4fe5-b1d4-98971601856d admin admin] PATH_INFO: `/v3/projects/a5ddfe4773c34b6182e786cc36c0ad8f/users/80bb1a6d238345cab97bf31cc60d789e/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:13.023076 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcac162-4e8c-4fe5-b1d4-98971601856d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ddfe4773c34b6182e786cc36c0ad8f, user_id=80bb1a6d238345cab97bf31cc60d789e, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:13.025844 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcac162-4e8c-4fe5-b1d4-98971601856d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:13.038966 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1395/2771] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:42:12 2022] PUT /identity/v3/projects/a5ddfe4773c34b6182e786cc36c0ad8f/users/80bb1a6d238345cab97bf31cc60d789e/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:13.050938 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a9755-1d90-442f-a48c-3ff5f7c1275e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:13.051222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a9755-1d90-442f-a48c-3ff5f7c1275e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:13.051501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a9755-1d90-442f-a48c-3ff5f7c1275e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:13.079020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0a9a9755-1d90-442f-a48c-3ff5f7c1275e None None] MFA Rules not processed for user `80bb1a6d238345cab97bf31cc60d789e`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:42:13.080256 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0a9a9755-1d90-442f-a48c-3ff5f7c1275e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:13.151754 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1377/2772] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:42:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:42:13.156507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcac162-4e8c-4fe5-b1d4-98971601856d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:13.220238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcac162-4e8c-4fe5-b1d4-98971601856d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:13.225034 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece9f55-6926-4c4d-ace0-aec872fb3938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:13.225625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9f55-6926-4c4d-ace0-aec872fb3938 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:13.225903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9f55-6926-4c4d-ace0-aec872fb3938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:13.226173 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9f55-6926-4c4d-ace0-aec872fb3938 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:13.226816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece9f55-6926-4c4d-ace0-aec872fb3938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:13.228741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece9f55-6926-4c4d-ace0-aec872fb3938 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:13.256346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1396/2773] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:42:13 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:42:13.260598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a9755-1d90-442f-a48c-3ff5f7c1275e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:13.271834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a9755-1d90-442f-a48c-3ff5f7c1275e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:13.327792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61789dc-c4c4-4b1e-bcf1-97f236481fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:13.328419 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61789dc-c4c4-4b1e-bcf1-97f236481fda admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:13.328702 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61789dc-c4c4-4b1e-bcf1-97f236481fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:13.328979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61789dc-c4c4-4b1e-bcf1-97f236481fda admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:13.329613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61789dc-c4c4-4b1e-bcf1-97f236481fda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:13.331531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61789dc-c4c4-4b1e-bcf1-97f236481fda admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:13.352963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1378/2774] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:42:13 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:42:13.357006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece9f55-6926-4c4d-ace0-aec872fb3938 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:13.368170 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece9f55-6926-4c4d-ace0-aec872fb3938 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:13.373336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423e71b-581c-478c-b1dc-9a13b667f88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:13.373929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423e71b-581c-478c-b1dc-9a13b667f88d admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:13.374209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423e71b-581c-478c-b1dc-9a13b667f88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:13.374481 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423e71b-581c-478c-b1dc-9a13b667f88d admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:13.375134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423e71b-581c-478c-b1dc-9a13b667f88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:13.376789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423e71b-581c-478c-b1dc-9a13b667f88d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:13.387153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1397/2775] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:42:13.392022 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61789dc-c4c4-4b1e-bcf1-97f236481fda admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:13.402123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61789dc-c4c4-4b1e-bcf1-97f236481fda admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:13.407125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555ce0c-4b9d-4a6f-abf8-108cc82ad5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:13.407777 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ce0c-4b9d-4a6f-abf8-108cc82ad5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:13.408081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ce0c-4b9d-4a6f-abf8-108cc82ad5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:13.408364 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ce0c-4b9d-4a6f-abf8-108cc82ad5cd admin admin] PATH_INFO: `/v3/projects/cd045b7fc924481190af98bccbaecde1/users/c62d013968f74789b3a4fc5ced880ae7/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:13.423303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555ce0c-4b9d-4a6f-abf8-108cc82ad5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd045b7fc924481190af98bccbaecde1, user_id=c62d013968f74789b3a4fc5ced880ae7, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:13.426267 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555ce0c-4b9d-4a6f-abf8-108cc82ad5cd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:13.439593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1379/2776] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:42:13 2022] PUT /identity/v3/projects/cd045b7fc924481190af98bccbaecde1/users/c62d013968f74789b3a4fc5ced880ae7/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:13.443499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423e71b-581c-478c-b1dc-9a13b667f88d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:13.454224 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423e71b-581c-478c-b1dc-9a13b667f88d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:13.510855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132edd31-8ae4-4219-9055-0c2b9f16cd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:13.511375 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132edd31-8ae4-4219-9055-0c2b9f16cd0e admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:13.511656 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132edd31-8ae4-4219-9055-0c2b9f16cd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:13.511929 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132edd31-8ae4-4219-9055-0c2b9f16cd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:13.512684 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132edd31-8ae4-4219-9055-0c2b9f16cd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:13.514330 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132edd31-8ae4-4219-9055-0c2b9f16cd0e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:13.523949 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1398/2777] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:42:13.528109 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555ce0c-4b9d-4a6f-abf8-108cc82ad5cd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:13.538556 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555ce0c-4b9d-4a6f-abf8-108cc82ad5cd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:13.543206 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563abc73-c8f8-4ca1-8e16-98b39affd5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:13.544110 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563abc73-c8f8-4ca1-8e16-98b39affd5ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:13.544405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563abc73-c8f8-4ca1-8e16-98b39affd5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:13.545224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563abc73-c8f8-4ca1-8e16-98b39affd5ce admin admin] PATH_INFO: `/v3/projects/cd045b7fc924481190af98bccbaecde1/users/c62d013968f74789b3a4fc5ced880ae7/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:13.547139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563abc73-c8f8-4ca1-8e16-98b39affd5ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd045b7fc924481190af98bccbaecde1, user_id=c62d013968f74789b3a4fc5ced880ae7, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:13.549818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563abc73-c8f8-4ca1-8e16-98b39affd5ce admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:13.552018 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:13.562607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1380/2778] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:42:13 2022] PUT /identity/v3/projects/cd045b7fc924481190af98bccbaecde1/users/c62d013968f74789b3a4fc5ced880ae7/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:13.572735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21341d68-b617-49bf-8fcd-536e1b047b11 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:13.573020 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21341d68-b617-49bf-8fcd-536e1b047b11 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:13.573295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21341d68-b617-49bf-8fcd-536e1b047b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:13.601919 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-21341d68-b617-49bf-8fcd-536e1b047b11 None None] MFA Rules not processed for user `c62d013968f74789b3a4fc5ced880ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:42:13.602634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-21341d68-b617-49bf-8fcd-536e1b047b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:13.613065 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:13.673162 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1399/2779] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:42:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:42:13.684234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563abc73-c8f8-4ca1-8e16-98b39affd5ce admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:13.686706 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-563abc73-c8f8-4ca1-8e16-98b39affd5ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:13.776311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563abc73-c8f8-4ca1-8e16-98b39affd5ce admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:13.779975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1c992-6682-4081-91cb-6f0143bfb4e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:13.780624 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1c992-6682-4081-91cb-6f0143bfb4e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:13.780904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1c992-6682-4081-91cb-6f0143bfb4e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:13.781183 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1c992-6682-4081-91cb-6f0143bfb4e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:13.790516 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1c992-6682-4081-91cb-6f0143bfb4e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:13.791952 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1c992-6682-4081-91cb-6f0143bfb4e7 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:13.802097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1381/2780] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:42:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:42:13.876718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21341d68-b617-49bf-8fcd-536e1b047b11 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:13.886189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21341d68-b617-49bf-8fcd-536e1b047b11 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:13.889735 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c3e00-6be0-4996-a60e-cfc2f5c69003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:13.890319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c3e00-6be0-4996-a60e-cfc2f5c69003 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:13.890601 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c3e00-6be0-4996-a60e-cfc2f5c69003 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:13.890869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c3e00-6be0-4996-a60e-cfc2f5c69003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:13.947960 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c3e00-6be0-4996-a60e-cfc2f5c69003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:13.949470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c3e00-6be0-4996-a60e-cfc2f5c69003 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:13.958397 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1400/2781] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:42:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:42:23.797936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=1,keystone:SELECT=41 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:23.954811 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=52 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:41.908066 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1c992-6682-4081-91cb-6f0143bfb4e7 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:41.909743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-28d1c992-6682-4081-91cb-6f0143bfb4e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:42.003821 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1c992-6682-4081-91cb-6f0143bfb4e7 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:42.009124 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a213c3-3e51-4ad1-b463-1f8db40d6176 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:42.009728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a213c3-3e51-4ad1-b463-1f8db40d6176 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:42.010013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a213c3-3e51-4ad1-b463-1f8db40d6176 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:42.010319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a213c3-3e51-4ad1-b463-1f8db40d6176 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:42.012190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a8a213c3-3e51-4ad1-b463-1f8db40d6176 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:42.023654 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a213c3-3e51-4ad1-b463-1f8db40d6176 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:42.025350 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a213c3-3e51-4ad1-b463-1f8db40d6176 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:42.035560 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1382/2782] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:42:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:42:52.031019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=41 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:55.448183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c3e00-6be0-4996-a60e-cfc2f5c69003 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:55.515677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c3e00-6be0-4996-a60e-cfc2f5c69003 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:55.520885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:55.521502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:55.521778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:55.522048 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] PATH_INFO: `/v3/users/80bb1a6d238345cab97bf31cc60d789e` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:55.523156 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80bb1a6d238345cab97bf31cc60d789e)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:55.525107 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:55.572204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80bb1a6d238345cab97bf31cc60d789e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:55.572786 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80bb1a6d238345cab97bf31cc60d789e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:55.573388 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80bb1a6d238345cab97bf31cc60d789e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:55.582263 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80bb1a6d238345cab97bf31cc60d789e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:55.589672 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80bb1a6d238345cab97bf31cc60d789e'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:55.607418 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1401/2783] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:42:55 2022] DELETE /identity/v3/users/80bb1a6d238345cab97bf31cc60d789e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:42:55.611923 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ae53-f5e5-4f5d-a09e-b1684d9e436b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:55.612404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ae53-f5e5-4f5d-a09e-b1684d9e436b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:55.612526 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ae53-f5e5-4f5d-a09e-b1684d9e436b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:55.632845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c006ae53-f5e5-4f5d-a09e-b1684d9e436b None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:42:55.634082 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c006ae53-f5e5-4f5d-a09e-b1684d9e436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:55.817241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1383/2784] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:42:55.827225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:55.828629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:55.886354 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5cd3c-691f-4877-9fec-fb2d2e760ba3 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:55.889932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2abf55-097c-460b-baad-283672005552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:55.890524 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2abf55-097c-460b-baad-283672005552 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:55.890805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2abf55-097c-460b-baad-283672005552 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:55.891075 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2abf55-097c-460b-baad-283672005552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:55.899882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2abf55-097c-460b-baad-283672005552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:55.901538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2abf55-097c-460b-baad-283672005552 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:55.910827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1402/2785] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:42:56.195004 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006ae53-f5e5-4f5d-a09e-b1684d9e436b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:56.196601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c006ae53-f5e5-4f5d-a09e-b1684d9e436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:56.209117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006ae53-f5e5-4f5d-a09e-b1684d9e436b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:56.213823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:56.214475 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:56.214751 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:56.215052 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] PATH_INFO: `/v3/projects/a5ddfe4773c34b6182e786cc36c0ad8f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:56.216205 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5ddfe4773c34b6182e786cc36c0ad8f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:56.218106 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:56.266294 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5ddfe4773c34b6182e786cc36c0ad8f', 'request_id': 'req-93f223c0-b4fa-40f9-9170-7b8697450bfc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.272420 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5ddfe4773c34b6182e786cc36c0ad8f', 'request_id': 'req-93f223c0-b4fa-40f9-9170-7b8697450bfc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.277842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5ddfe4773c34b6182e786cc36c0ad8f', 'request_id': 'req-93f223c0-b4fa-40f9-9170-7b8697450bfc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.291409 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] The token cache is being invalidate because project a5ddfe4773c34b6182e786cc36c0ad8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:42:56.291694 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.292348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.294067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1384/2786] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:42:56 2022] DELETE /identity/v3/projects/a5ddfe4773c34b6182e786cc36c0ad8f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:56.298026 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2abf55-097c-460b-baad-283672005552 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:56.308069 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2abf55-097c-460b-baad-283672005552 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:56.325702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:56.366951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75235a5-bd27-4d7f-b855-89f4e091f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:56.367474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75235a5-bd27-4d7f-b855-89f4e091f887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:56.367757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75235a5-bd27-4d7f-b855-89f4e091f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:56.368028 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75235a5-bd27-4d7f-b855-89f4e091f887 admin admin] PATH_INFO: `/v3/users/c62d013968f74789b3a4fc5ced880ae7` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:56.369208 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75235a5-bd27-4d7f-b855-89f4e091f887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c62d013968f74789b3a4fc5ced880ae7)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:56.371127 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75235a5-bd27-4d7f-b855-89f4e091f887 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:56.417816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b75235a5-bd27-4d7f-b855-89f4e091f887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c62d013968f74789b3a4fc5ced880ae7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.418745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b75235a5-bd27-4d7f-b855-89f4e091f887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c62d013968f74789b3a4fc5ced880ae7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.419362 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b75235a5-bd27-4d7f-b855-89f4e091f887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c62d013968f74789b3a4fc5ced880ae7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.427078 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b75235a5-bd27-4d7f-b855-89f4e091f887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c62d013968f74789b3a4fc5ced880ae7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.435445 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b75235a5-bd27-4d7f-b855-89f4e091f887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c62d013968f74789b3a4fc5ced880ae7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.468230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1403/2787] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:42:56 2022] DELETE /identity/v3/users/c62d013968f74789b3a4fc5ced880ae7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:42:56.777198 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:56.779374 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:56.800470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f223c0-b4fa-40f9-9170-7b8697450bfc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:56.864611 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:56.865241 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:56.865520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:56.865791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] PATH_INFO: `/v3/projects/cd045b7fc924481190af98bccbaecde1` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:56.866914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd045b7fc924481190af98bccbaecde1)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:56.868870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:56.902776 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:56.920974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd045b7fc924481190af98bccbaecde1', 'request_id': 'req-25e37a75-df36-4022-9bb7-f4c9ea48490a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.927466 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd045b7fc924481190af98bccbaecde1', 'request_id': 'req-25e37a75-df36-4022-9bb7-f4c9ea48490a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.933395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd045b7fc924481190af98bccbaecde1', 'request_id': 'req-25e37a75-df36-4022-9bb7-f4c9ea48490a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.946421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] The token cache is being invalidate because project cd045b7fc924481190af98bccbaecde1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:42:56.946711 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.947342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-25e37a75-df36-4022-9bb7-f4c9ea48490a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:56.949123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1385/2788] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:42:56 2022] DELETE /identity/v3/projects/cd045b7fc924481190af98bccbaecde1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:56.954620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f44aab-f4f1-419e-a725-5009c9070ab8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:56.954935 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f44aab-f4f1-419e-a725-5009c9070ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:56.955948 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f44aab-f4f1-419e-a725-5009c9070ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:56.974944 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-88f44aab-f4f1-419e-a725-5009c9070ab8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:42:56.976158 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-88f44aab-f4f1-419e-a725-5009c9070ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:57.035521 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1404/2789] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:42:57.100247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6840f-7042-46f8-823f-e5b4efb3f8fd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.100530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6840f-7042-46f8-823f-e5b4efb3f8fd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.100802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6840f-7042-46f8-823f-e5b4efb3f8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.121505 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-38d6840f-7042-46f8-823f-e5b4efb3f8fd None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:42:57.122712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-38d6840f-7042-46f8-823f-e5b4efb3f8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:57.131130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1386/2790] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:42:57.135305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f44aab-f4f1-419e-a725-5009c9070ab8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.146859 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f44aab-f4f1-419e-a725-5009c9070ab8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.150454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d375f-27bc-431d-95f9-14a4d62f9fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.151041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d375f-27bc-431d-95f9-14a4d62f9fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.151317 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d375f-27bc-431d-95f9-14a4d62f9fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.151586 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d375f-27bc-431d-95f9-14a4d62f9fdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.152255 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d375f-27bc-431d-95f9-14a4d62f9fdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:42:57.152554 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d375f-27bc-431d-95f9-14a4d62f9fdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.154137 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d375f-27bc-431d-95f9-14a4d62f9fdf admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.165766 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1405/2791] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:42:57.170482 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6840f-7042-46f8-823f-e5b4efb3f8fd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.179732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6840f-7042-46f8-823f-e5b4efb3f8fd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.184431 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794d4fb-c666-48e5-90b7-0838e9d7eda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.185023 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794d4fb-c666-48e5-90b7-0838e9d7eda5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.185304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794d4fb-c666-48e5-90b7-0838e9d7eda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.185607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794d4fb-c666-48e5-90b7-0838e9d7eda5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.186280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794d4fb-c666-48e5-90b7-0838e9d7eda5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.188315 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794d4fb-c666-48e5-90b7-0838e9d7eda5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.217112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1387/2792] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:42:57 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:42:57.221335 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d375f-27bc-431d-95f9-14a4d62f9fdf admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.231343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d375f-27bc-431d-95f9-14a4d62f9fdf admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.259578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:57.295295 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde11a0-ddd7-4598-b031-5bd95d96611e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.295900 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde11a0-ddd7-4598-b031-5bd95d96611e admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.296225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde11a0-ddd7-4598-b031-5bd95d96611e admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.296502 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde11a0-ddd7-4598-b031-5bd95d96611e admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.297130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde11a0-ddd7-4598-b031-5bd95d96611e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.299583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde11a0-ddd7-4598-b031-5bd95d96611e admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.321210 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1406/2793] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:42:57 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:42:57.325312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794d4fb-c666-48e5-90b7-0838e9d7eda5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.335635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794d4fb-c666-48e5-90b7-0838e9d7eda5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.340936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76f616-265e-4ae7-ae49-b06dd4bf24b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.341540 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76f616-265e-4ae7-ae49-b06dd4bf24b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.341823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76f616-265e-4ae7-ae49-b06dd4bf24b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.342097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76f616-265e-4ae7-ae49-b06dd4bf24b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.342804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76f616-265e-4ae7-ae49-b06dd4bf24b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.344372 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76f616-265e-4ae7-ae49-b06dd4bf24b3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.354401 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1388/2794] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:42:57.358341 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde11a0-ddd7-4598-b031-5bd95d96611e admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.368681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde11a0-ddd7-4598-b031-5bd95d96611e admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.373535 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a1be0-fda6-4a9c-a508-60a410b66d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.374129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a1be0-fda6-4a9c-a508-60a410b66d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.374416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a1be0-fda6-4a9c-a508-60a410b66d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.374688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a1be0-fda6-4a9c-a508-60a410b66d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.375325 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a1be0-fda6-4a9c-a508-60a410b66d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.377627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a1be0-fda6-4a9c-a508-60a410b66d4c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.388006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1407/2795] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:42:57.392095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76f616-265e-4ae7-ae49-b06dd4bf24b3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.402270 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76f616-265e-4ae7-ae49-b06dd4bf24b3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.406924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f1113-0286-4002-a6a6-b6a977f8162b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.407561 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f1113-0286-4002-a6a6-b6a977f8162b admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.407845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f1113-0286-4002-a6a6-b6a977f8162b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.408217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f1113-0286-4002-a6a6-b6a977f8162b admin admin] PATH_INFO: `/v3/projects/e39b94857e3240dea8f1282149cd28f3/users/1ddc765ab33b46f8890e3842b3a52869/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.422691 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f1113-0286-4002-a6a6-b6a977f8162b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39b94857e3240dea8f1282149cd28f3, user_id=1ddc765ab33b46f8890e3842b3a52869, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.425854 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f1113-0286-4002-a6a6-b6a977f8162b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.440003 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1389/2796] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:42:57 2022] PUT /identity/v3/projects/e39b94857e3240dea8f1282149cd28f3/users/1ddc765ab33b46f8890e3842b3a52869/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:57.449867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318bacf-f2e2-4b9f-a538-43d98d16d4b8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.450154 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318bacf-f2e2-4b9f-a538-43d98d16d4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.450425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318bacf-f2e2-4b9f-a538-43d98d16d4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.478545 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-a318bacf-f2e2-4b9f-a538-43d98d16d4b8 None None] MFA Rules not processed for user `1ddc765ab33b46f8890e3842b3a52869`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:42:57.479798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a318bacf-f2e2-4b9f-a538-43d98d16d4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:57.543057 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1408/2797] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:42:57.547813 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f1113-0286-4002-a6a6-b6a977f8162b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.609927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f1113-0286-4002-a6a6-b6a977f8162b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.614576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cce338-94a6-4e3f-9dc5-477381b51df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.615200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cce338-94a6-4e3f-9dc5-477381b51df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.615481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cce338-94a6-4e3f-9dc5-477381b51df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.615758 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cce338-94a6-4e3f-9dc5-477381b51df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.616422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cce338-94a6-4e3f-9dc5-477381b51df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.618313 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cce338-94a6-4e3f-9dc5-477381b51df1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.646112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1390/2798] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:42:57 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:42:57.650498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318bacf-f2e2-4b9f-a538-43d98d16d4b8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.661119 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318bacf-f2e2-4b9f-a538-43d98d16d4b8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.719037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aaf171-dcf5-4000-90db-86f872d5ef93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.719634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaf171-dcf5-4000-90db-86f872d5ef93 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.719913 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaf171-dcf5-4000-90db-86f872d5ef93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.720220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaf171-dcf5-4000-90db-86f872d5ef93 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.720883 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aaf171-dcf5-4000-90db-86f872d5ef93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.722798 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aaf171-dcf5-4000-90db-86f872d5ef93 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.743172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1409/2799] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:42:57 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:42:57.747563 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cce338-94a6-4e3f-9dc5-477381b51df1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.758511 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cce338-94a6-4e3f-9dc5-477381b51df1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.763228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a2f40-4996-4c47-ac17-b9f01b7092ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.763825 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a2f40-4996-4c47-ac17-b9f01b7092ac admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.764159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a2f40-4996-4c47-ac17-b9f01b7092ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.764514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a2f40-4996-4c47-ac17-b9f01b7092ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.765087 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a2f40-4996-4c47-ac17-b9f01b7092ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.766734 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a2f40-4996-4c47-ac17-b9f01b7092ac admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.777759 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1391/2800] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:42:57.781972 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aaf171-dcf5-4000-90db-86f872d5ef93 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.792195 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aaf171-dcf5-4000-90db-86f872d5ef93 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.797146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd08b89-9855-4e9f-9748-8140f0ed1e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.797792 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd08b89-9855-4e9f-9748-8140f0ed1e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.798066 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd08b89-9855-4e9f-9748-8140f0ed1e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.798336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd08b89-9855-4e9f-9748-8140f0ed1e83 admin admin] PATH_INFO: `/v3/projects/d1e0cb65f25148928febdc48234472e8/users/edecca1f1de84a308eb96214cd7a9a30/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.812637 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd08b89-9855-4e9f-9748-8140f0ed1e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1e0cb65f25148928febdc48234472e8, user_id=edecca1f1de84a308eb96214cd7a9a30, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.815769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd08b89-9855-4e9f-9748-8140f0ed1e83 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.828848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1410/2801] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:42:57 2022] PUT /identity/v3/projects/d1e0cb65f25148928febdc48234472e8/users/edecca1f1de84a308eb96214cd7a9a30/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:57.833422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a2f40-4996-4c47-ac17-b9f01b7092ac admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.843310 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a2f40-4996-4c47-ac17-b9f01b7092ac admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.863570 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:57.902948 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c253e-cc8f-46c4-9437-ca64e0213dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.903564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c253e-cc8f-46c4-9437-ca64e0213dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.903842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c253e-cc8f-46c4-9437-ca64e0213dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.904145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c253e-cc8f-46c4-9437-ca64e0213dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.904818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c253e-cc8f-46c4-9437-ca64e0213dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.906393 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c253e-cc8f-46c4-9437-ca64e0213dc8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.916430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1392/2802] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:42:57.920863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd08b89-9855-4e9f-9748-8140f0ed1e83 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:57.926395 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:57.932939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd08b89-9855-4e9f-9748-8140f0ed1e83 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:57.942800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe94c7-33c6-49c1-91c3-f03cd42d3467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:57.943454 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe94c7-33c6-49c1-91c3-f03cd42d3467 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.943731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe94c7-33c6-49c1-91c3-f03cd42d3467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.944001 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe94c7-33c6-49c1-91c3-f03cd42d3467 admin admin] PATH_INFO: `/v3/projects/d1e0cb65f25148928febdc48234472e8/users/edecca1f1de84a308eb96214cd7a9a30/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.945959 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe94c7-33c6-49c1-91c3-f03cd42d3467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1e0cb65f25148928febdc48234472e8, user_id=edecca1f1de84a308eb96214cd7a9a30, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:57.948696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe94c7-33c6-49c1-91c3-f03cd42d3467 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:57.959445 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1411/2803] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:42:57 2022] PUT /identity/v3/projects/d1e0cb65f25148928febdc48234472e8/users/edecca1f1de84a308eb96214cd7a9a30/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:57.970220 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c023f3-f49a-462a-983f-3fe7659fa2a1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:57.970539 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c023f3-f49a-462a-983f-3fe7659fa2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:57.970843 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c023f3-f49a-462a-983f-3fe7659fa2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:57.998990 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-97c023f3-f49a-462a-983f-3fe7659fa2a1 None None] MFA Rules not processed for user `edecca1f1de84a308eb96214cd7a9a30`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:42:58.000228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-97c023f3-f49a-462a-983f-3fe7659fa2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:58.066222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1393/2804] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:42:58.079340 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe94c7-33c6-49c1-91c3-f03cd42d3467 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:58.081369 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c1fe94c7-33c6-49c1-91c3-f03cd42d3467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:58.138969 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe94c7-33c6-49c1-91c3-f03cd42d3467 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:58.142633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea7f8c-cc03-492d-96d0-174e5e6edf86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:58.143226 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea7f8c-cc03-492d-96d0-174e5e6edf86 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:58.143504 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea7f8c-cc03-492d-96d0-174e5e6edf86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:58.143776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea7f8c-cc03-492d-96d0-174e5e6edf86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:58.152677 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea7f8c-cc03-492d-96d0-174e5e6edf86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:58.154103 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea7f8c-cc03-492d-96d0-174e5e6edf86 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:58.163331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1412/2805] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:42:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:42:58.467929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c023f3-f49a-462a-983f-3fe7659fa2a1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:58.477553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c023f3-f49a-462a-983f-3fe7659fa2a1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:58.483494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80900fdf-12dc-4464-b1cf-8656328b3412 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edecca1f1de84a308eb96214cd7a9a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1e0cb65f25148928febdc48234472e8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:58.484187 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80900fdf-12dc-4464-b1cf-8656328b3412 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:58.485375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80900fdf-12dc-4464-b1cf-8656328b3412 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:58.485375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80900fdf-12dc-4464-b1cf-8656328b3412 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:58.485375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80900fdf-12dc-4464-b1cf-8656328b3412 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:42:58.486860 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80900fdf-12dc-4464-b1cf-8656328b3412 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:58.487873 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80900fdf-12dc-4464-b1cf-8656328b3412 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:58.499544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1394/2806] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:42:58.504718 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea7f8c-cc03-492d-96d0-174e5e6edf86 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:58.514194 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea7f8c-cc03-492d-96d0-174e5e6edf86 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:58.519401 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808e06e-b5f1-4916-9b36-1326b5804260 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edecca1f1de84a308eb96214cd7a9a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1e0cb65f25148928febdc48234472e8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:58.519992 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808e06e-b5f1-4916-9b36-1326b5804260 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:58.520332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808e06e-b5f1-4916-9b36-1326b5804260 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:58.520608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808e06e-b5f1-4916-9b36-1326b5804260 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:58.521228 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808e06e-b5f1-4916-9b36-1326b5804260 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:58.523130 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808e06e-b5f1-4916-9b36-1326b5804260 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:58.549463 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1413/2807] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:42:58 2022] POST /identity/v3/projects => generated 353 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:42:58.553661 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80900fdf-12dc-4464-b1cf-8656328b3412 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:58.563432 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80900fdf-12dc-4464-b1cf-8656328b3412 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:58.617749 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff742d-cdb3-412c-87b4-3f78da24c51e tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edecca1f1de84a308eb96214cd7a9a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1e0cb65f25148928febdc48234472e8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:58.618344 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff742d-cdb3-412c-87b4-3f78da24c51e tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:58.618628 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff742d-cdb3-412c-87b4-3f78da24c51e tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:58.618904 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff742d-cdb3-412c-87b4-3f78da24c51e tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:58.619541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff742d-cdb3-412c-87b4-3f78da24c51e tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:42:58.619931 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff742d-cdb3-412c-87b4-3f78da24c51e tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:58.621545 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff742d-cdb3-412c-87b4-3f78da24c51e tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:58.633286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1395/2808] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:42:58.669313 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808e06e-b5f1-4916-9b36-1326b5804260 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:58.679469 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808e06e-b5f1-4916-9b36-1326b5804260 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:58.683893 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edecca1f1de84a308eb96214cd7a9a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1e0cb65f25148928febdc48234472e8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:58.685272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:58.685272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:58.685272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] PATH_INFO: `/v3/projects/8abc9654317b442eb45fe3fe817ab096` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:58.686179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8abc9654317b442eb45fe3fe817ab096)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:58.688074 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:58.734420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8abc9654317b442eb45fe3fe817ab096', 'request_id': 'req-b7690d79-3996-44e9-a8b8-af20b3ae5b72'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:58.741453 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8abc9654317b442eb45fe3fe817ab096', 'request_id': 'req-b7690d79-3996-44e9-a8b8-af20b3ae5b72'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:58.747588 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8abc9654317b442eb45fe3fe817ab096', 'request_id': 'req-b7690d79-3996-44e9-a8b8-af20b3ae5b72'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:58.759343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] The token cache is being invalidate because project 8abc9654317b442eb45fe3fe817ab096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:42:58.759635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:58.760304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-b7690d79-3996-44e9-a8b8-af20b3ae5b72 tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:58.762032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1414/2809] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:42:58 2022] DELETE /identity/v3/projects/8abc9654317b442eb45fe3fe817ab096 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:58.961916 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff742d-cdb3-412c-87b4-3f78da24c51e tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:58.963745 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0fff742d-cdb3-412c-87b4-3f78da24c51e tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:58.994197 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:59.028866 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff742d-cdb3-412c-87b4-3f78da24c51e tempest-VolumeQuotaClassesTest-545405582 tempest-VolumeQuotaClassesTest-545405582-project-admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:59.084122 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:59.084741 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:59.085018 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:59.085287 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] PATH_INFO: `/v3/users/1ddc765ab33b46f8890e3842b3a52869` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:59.086395 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ddc765ab33b46f8890e3842b3a52869)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:59.088328 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:59.134425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ddc765ab33b46f8890e3842b3a52869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.135312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ddc765ab33b46f8890e3842b3a52869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.136147 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ddc765ab33b46f8890e3842b3a52869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.145346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ddc765ab33b46f8890e3842b3a52869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.152342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ddc765ab33b46f8890e3842b3a52869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.171012 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1396/2810] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:42:58 2022] DELETE /identity/v3/users/1ddc765ab33b46f8890e3842b3a52869 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:59.175496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53ecfa-ae9d-4b50-a209-775feccdaef1 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:59.175865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53ecfa-ae9d-4b50-a209-775feccdaef1 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:59.176176 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53ecfa-ae9d-4b50-a209-775feccdaef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:59.195459 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-1e53ecfa-ae9d-4b50-a209-775feccdaef1 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:42:59.196607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1e53ecfa-ae9d-4b50-a209-775feccdaef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:59.243629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:59.258089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1415/2811] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:42:59.268191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:59.269689 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:59.326928 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb6bc4-6924-4d83-ad27-62992fb4da96 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:59.330573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec08c53-9ced-4630-9715-ffad1f4b7d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:59.331167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec08c53-9ced-4630-9715-ffad1f4b7d32 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:59.331446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec08c53-9ced-4630-9715-ffad1f4b7d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:59.331720 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec08c53-9ced-4630-9715-ffad1f4b7d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:59.338565 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:59.341043 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec08c53-9ced-4630-9715-ffad1f4b7d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:59.342966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec08c53-9ced-4630-9715-ffad1f4b7d32 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:59.354293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1397/2812] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:42:59.622301 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53ecfa-ae9d-4b50-a209-775feccdaef1 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:59.623775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1e53ecfa-ae9d-4b50-a209-775feccdaef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:42:59.635583 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53ecfa-ae9d-4b50-a209-775feccdaef1 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:59.640236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:59.640867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:59.641145 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:59.641416 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] PATH_INFO: `/v3/projects/e39b94857e3240dea8f1282149cd28f3` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:59.643062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39b94857e3240dea8f1282149cd28f3)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:59.645036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:59.691713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39b94857e3240dea8f1282149cd28f3', 'request_id': 'req-3a84b369-776e-4008-a15e-0d4b13110f9d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.698094 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39b94857e3240dea8f1282149cd28f3', 'request_id': 'req-3a84b369-776e-4008-a15e-0d4b13110f9d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.703770 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39b94857e3240dea8f1282149cd28f3', 'request_id': 'req-3a84b369-776e-4008-a15e-0d4b13110f9d'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.717776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] The token cache is being invalidate because project e39b94857e3240dea8f1282149cd28f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:42:59.718061 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.718680 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.720172 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1416/2813] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:42:59 2022] DELETE /identity/v3/projects/e39b94857e3240dea8f1282149cd28f3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:42:59.724953 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec08c53-9ced-4630-9715-ffad1f4b7d32 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:42:59.734921 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec08c53-9ced-4630-9715-ffad1f4b7d32 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:42:59.791573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa27984-0f69-4092-bf81-c7723f2f41f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:42:59.792384 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa27984-0f69-4092-bf81-c7723f2f41f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:42:59.792669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa27984-0f69-4092-bf81-c7723f2f41f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:42:59.792948 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa27984-0f69-4092-bf81-c7723f2f41f0 admin admin] PATH_INFO: `/v3/users/edecca1f1de84a308eb96214cd7a9a30` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:42:59.794346 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa27984-0f69-4092-bf81-c7723f2f41f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edecca1f1de84a308eb96214cd7a9a30)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:42:59.796314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa27984-0f69-4092-bf81-c7723f2f41f0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:42:59.841134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9fa27984-0f69-4092-bf81-c7723f2f41f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edecca1f1de84a308eb96214cd7a9a30'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.841779 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9fa27984-0f69-4092-bf81-c7723f2f41f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edecca1f1de84a308eb96214cd7a9a30'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.842383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9fa27984-0f69-4092-bf81-c7723f2f41f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edecca1f1de84a308eb96214cd7a9a30'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.850184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9fa27984-0f69-4092-bf81-c7723f2f41f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edecca1f1de84a308eb96214cd7a9a30'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.857469 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9fa27984-0f69-4092-bf81-c7723f2f41f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edecca1f1de84a308eb96214cd7a9a30'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:42:59.874210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1398/2814] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:42:59 2022] DELETE /identity/v3/users/edecca1f1de84a308eb96214cd7a9a30 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:43:00.146347 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:00.147795 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:43:00.160723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84b369-776e-4008-a15e-0d4b13110f9d admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:00.216446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:00.217063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:00.217340 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:00.217610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] PATH_INFO: `/v3/projects/d1e0cb65f25148928febdc48234472e8` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:00.218697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1e0cb65f25148928febdc48234472e8)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:00.220667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:00.266133 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1e0cb65f25148928febdc48234472e8', 'request_id': 'req-87c85a44-809d-4db5-aef7-e8238779c935'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:00.272019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1e0cb65f25148928febdc48234472e8', 'request_id': 'req-87c85a44-809d-4db5-aef7-e8238779c935'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:00.277784 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1e0cb65f25148928febdc48234472e8', 'request_id': 'req-87c85a44-809d-4db5-aef7-e8238779c935'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:00.289474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] The token cache is being invalidate because project d1e0cb65f25148928febdc48234472e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:43:00.289756 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:00.290380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-87c85a44-809d-4db5-aef7-e8238779c935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:00.292073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1417/2815] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:43:00 2022] DELETE /identity/v3/projects/d1e0cb65f25148928febdc48234472e8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:43:00.298331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f6e6e-cdae-4d23-af33-26ae88b6814d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:00.299475 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f6e6e-cdae-4d23-af33-26ae88b6814d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:00.299475 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f6e6e-cdae-4d23-af33-26ae88b6814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:00.319544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-892f6e6e-cdae-4d23-af33-26ae88b6814d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:43:00.320774 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-892f6e6e-cdae-4d23-af33-26ae88b6814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:43:00.380228 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1399/2816] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:43:00.388052 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d9ad-4ad5-4a4c-830d-09738c78a8a7 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:00.388363 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d9ad-4ad5-4a4c-830d-09738c78a8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:00.388635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d9ad-4ad5-4a4c-830d-09738c78a8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:00.402532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:00.408233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-4cb6d9ad-4ad5-4a4c-830d-09738c78a8a7 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:43:00.410144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-4cb6d9ad-4ad5-4a4c-830d-09738c78a8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:43:00.418463 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1418/2817] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:43:00.422522 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f6e6e-cdae-4d23-af33-26ae88b6814d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:00.432762 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f6e6e-cdae-4d23-af33-26ae88b6814d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:00.437427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9fdcf-db16-41b9-bc52-0c4e055ffedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:00.438017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9fdcf-db16-41b9-bc52-0c4e055ffedc admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:00.438299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9fdcf-db16-41b9-bc52-0c4e055ffedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:00.438574 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9fdcf-db16-41b9-bc52-0c4e055ffedc admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:00.439294 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9fdcf-db16-41b9-bc52-0c4e055ffedc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:43:00.440137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9fdcf-db16-41b9-bc52-0c4e055ffedc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:00.441728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9fdcf-db16-41b9-bc52-0c4e055ffedc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:00.453200 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1400/2818] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:43:00.457496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6d9ad-4ad5-4a4c-830d-09738c78a8a7 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:00.466729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6d9ad-4ad5-4a4c-830d-09738c78a8a7 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:00.471419 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512d37a-15ce-46c8-a044-a8ce6996dbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:00.472006 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512d37a-15ce-46c8-a044-a8ce6996dbef admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:00.472345 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512d37a-15ce-46c8-a044-a8ce6996dbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:00.472620 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512d37a-15ce-46c8-a044-a8ce6996dbef admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:00.473238 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512d37a-15ce-46c8-a044-a8ce6996dbef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:00.475128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512d37a-15ce-46c8-a044-a8ce6996dbef admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:00.504507 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1419/2819] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:43:00 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:43:00.509083 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9fdcf-db16-41b9-bc52-0c4e055ffedc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:00.514257 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:00.520807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9fdcf-db16-41b9-bc52-0c4e055ffedc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:00.579162 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb9e32-baae-4755-afe9-a5ca3c2c0fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:00.579790 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb9e32-baae-4755-afe9-a5ca3c2c0fff admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:00.580161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb9e32-baae-4755-afe9-a5ca3c2c0fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:00.580497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb9e32-baae-4755-afe9-a5ca3c2c0fff admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:00.581125 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb9e32-baae-4755-afe9-a5ca3c2c0fff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:00.583184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb9e32-baae-4755-afe9-a5ca3c2c0fff admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:00.605389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1401/2820] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:43:00 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:43:00.609774 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512d37a-15ce-46c8-a044-a8ce6996dbef admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:00.620352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512d37a-15ce-46c8-a044-a8ce6996dbef admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:00.624827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56629efb-9925-4966-9167-ffd34d63f33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:00.625428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629efb-9925-4966-9167-ffd34d63f33b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:00.625702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629efb-9925-4966-9167-ffd34d63f33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:00.625977 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629efb-9925-4966-9167-ffd34d63f33b admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:00.626610 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56629efb-9925-4966-9167-ffd34d63f33b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:00.628892 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56629efb-9925-4966-9167-ffd34d63f33b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:00.638484 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1420/2821] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:43:00.642312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb9e32-baae-4755-afe9-a5ca3c2c0fff admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:00.652058 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb9e32-baae-4755-afe9-a5ca3c2c0fff admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:00.656769 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39499c35-de84-4a06-a8fd-e347eb3ec658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:00.657356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499c35-de84-4a06-a8fd-e347eb3ec658 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:00.657646 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499c35-de84-4a06-a8fd-e347eb3ec658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:00.658478 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499c35-de84-4a06-a8fd-e347eb3ec658 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:00.659163 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39499c35-de84-4a06-a8fd-e347eb3ec658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:00.660850 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39499c35-de84-4a06-a8fd-e347eb3ec658 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:00.670918 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1402/2822] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:43:00.675541 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56629efb-9925-4966-9167-ffd34d63f33b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:00.685287 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56629efb-9925-4966-9167-ffd34d63f33b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:00.689808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eff5e-a2d1-4943-a90b-c85d63be6f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:00.690433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eff5e-a2d1-4943-a90b-c85d63be6f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:00.690709 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eff5e-a2d1-4943-a90b-c85d63be6f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:00.690978 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eff5e-a2d1-4943-a90b-c85d63be6f9f admin admin] PATH_INFO: `/v3/projects/f3756540754442a1a8b57d6dbcb23be8/users/e47e6865639b44f19c9d5e341bce8655/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:00.705635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816eff5e-a2d1-4943-a90b-c85d63be6f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3756540754442a1a8b57d6dbcb23be8, user_id=e47e6865639b44f19c9d5e341bce8655, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:00.708405 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816eff5e-a2d1-4943-a90b-c85d63be6f9f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:00.721537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1421/2823] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:43:00 2022] PUT /identity/v3/projects/f3756540754442a1a8b57d6dbcb23be8/users/e47e6865639b44f19c9d5e341bce8655/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:43:00.731201 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d8e8b-64b7-4aa9-941d-f2f4d1040d10 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:00.731483 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d8e8b-64b7-4aa9-941d-f2f4d1040d10 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:00.731756 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d8e8b-64b7-4aa9-941d-f2f4d1040d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:00.759191 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-973d8e8b-64b7-4aa9-941d-f2f4d1040d10 None None] MFA Rules not processed for user `e47e6865639b44f19c9d5e341bce8655`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:43:00.760427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-973d8e8b-64b7-4aa9-941d-f2f4d1040d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:43:00.824497 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1403/2824] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:43:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:43:00.835753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eff5e-a2d1-4943-a90b-c85d63be6f9f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:00.837471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-816eff5e-a2d1-4943-a90b-c85d63be6f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:43:00.920907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eff5e-a2d1-4943-a90b-c85d63be6f9f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:00.925192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44790d6d-f3e5-4b03-ba04-0df72b6e54db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:00.925794 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44790d6d-f3e5-4b03-ba04-0df72b6e54db service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:00.926073 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44790d6d-f3e5-4b03-ba04-0df72b6e54db service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:00.926344 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44790d6d-f3e5-4b03-ba04-0df72b6e54db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:00.934848 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44790d6d-f3e5-4b03-ba04-0df72b6e54db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:00.936348 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44790d6d-f3e5-4b03-ba04-0df72b6e54db service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:00.945023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1422/2825] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:43:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:43:10.821635 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:10.941082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:14.154950 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d8e8b-64b7-4aa9-941d-f2f4d1040d10 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:14.156764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-973d8e8b-64b7-4aa9-941d-f2f4d1040d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:43:14.241501 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d8e8b-64b7-4aa9-941d-f2f4d1040d10 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:14.247481 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82529d-bd9b-4d9c-996b-acd3eeed6804 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98e11312493643b199fca04cb2d77853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:14.248936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82529d-bd9b-4d9c-996b-acd3eeed6804 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:14.249238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82529d-bd9b-4d9c-996b-acd3eeed6804 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:14.249644 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82529d-bd9b-4d9c-996b-acd3eeed6804 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:14.251425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ae82529d-bd9b-4d9c-996b-acd3eeed6804 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:43:14.265402 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82529d-bd9b-4d9c-996b-acd3eeed6804 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:14.267336 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82529d-bd9b-4d9c-996b-acd3eeed6804 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:14.278847 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1404/2826] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:43:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:43:18.992392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52debf2-8d67-42e5-a67d-44ac513553ab None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:18.992918 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52debf2-8d67-42e5-a67d-44ac513553ab None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:18.992918 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52debf2-8d67-42e5-a67d-44ac513553ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:19.012269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d52debf2-8d67-42e5-a67d-44ac513553ab None None] MFA Rules not processed for user `311b0a8b22174a278793537963b5d142`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:43:19.013456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d52debf2-8d67-42e5-a67d-44ac513553ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:43:19.067253 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1423/2827] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:43:21.677818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82529d-bd9b-4d9c-996b-acd3eeed6804 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:21.752605 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82529d-bd9b-4d9c-996b-acd3eeed6804 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:21.757217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:21.757845 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:21.758128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:21.758401 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] PATH_INFO: `/v3/users/e47e6865639b44f19c9d5e341bce8655` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:21.759520 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47e6865639b44f19c9d5e341bce8655)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:21.761529 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:21.819234 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47e6865639b44f19c9d5e341bce8655'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:21.819927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47e6865639b44f19c9d5e341bce8655'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:21.820911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47e6865639b44f19c9d5e341bce8655'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:21.828855 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47e6865639b44f19c9d5e341bce8655'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:21.836763 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47e6865639b44f19c9d5e341bce8655'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:21.850061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:21.855362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1405/2828] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:43:21 2022] DELETE /identity/v3/users/e47e6865639b44f19c9d5e341bce8655 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:43:21.860000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897f793-f9f2-4217-a7fb-cd89ff23de23 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:21.860602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897f793-f9f2-4217-a7fb-cd89ff23de23 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:21.860719 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897f793-f9f2-4217-a7fb-cd89ff23de23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:21.879814 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2897f793-f9f2-4217-a7fb-cd89ff23de23 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:43:21.881147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2897f793-f9f2-4217-a7fb-cd89ff23de23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:43:21.946748 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1424/2829] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:43:21.962456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:21.964610 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:43:22.030378 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f4f1c-1690-49ae-8876-30ca1352a854 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:22.035863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d497fff-97ea-42f2-a7fd-67ec3cd17013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:22.036519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d497fff-97ea-42f2-a7fd-67ec3cd17013 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:22.036818 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d497fff-97ea-42f2-a7fd-67ec3cd17013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:22.037122 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d497fff-97ea-42f2-a7fd-67ec3cd17013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:22.047901 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d497fff-97ea-42f2-a7fd-67ec3cd17013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:22.049618 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d497fff-97ea-42f2-a7fd-67ec3cd17013 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:22.068910 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1406/2830] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:43:22.715412 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897f793-f9f2-4217-a7fb-cd89ff23de23 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:43:22.717016 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2897f793-f9f2-4217-a7fb-cd89ff23de23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:43:22.730129 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897f793-f9f2-4217-a7fb-cd89ff23de23 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:43:22.736732 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:43:22.738144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:43:22.738642 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:43:22.738920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] PATH_INFO: `/v3/projects/f3756540754442a1a8b57d6dbcb23be8` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:43:22.740478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3756540754442a1a8b57d6dbcb23be8)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:43:22.742281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:43:22.801311 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3756540754442a1a8b57d6dbcb23be8', 'request_id': 'req-79303883-48d0-4e96-a2e9-9c17bc62b6c7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:22.807675 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3756540754442a1a8b57d6dbcb23be8', 'request_id': 'req-79303883-48d0-4e96-a2e9-9c17bc62b6c7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:22.813336 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3756540754442a1a8b57d6dbcb23be8', 'request_id': 'req-79303883-48d0-4e96-a2e9-9c17bc62b6c7'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:22.826209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] The token cache is being invalidate because project f3756540754442a1a8b57d6dbcb23be8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:43:22.826499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:22.827142 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:43:22.828663 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1425/2831] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:43:22 2022] DELETE /identity/v3/projects/f3756540754442a1a8b57d6dbcb23be8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:43:32.056427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=27 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:32.823093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:27.832184 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d497fff-97ea-42f2-a7fd-67ec3cd17013 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:27.833601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7d497fff-97ea-42f2-a7fd-67ec3cd17013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:27.901346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:27.902764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:27.940523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d497fff-97ea-42f2-a7fd-67ec3cd17013 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:27.995058 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79303883-48d0-4e96-a2e9-9c17bc62b6c7 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:28.000112 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911503c3-920d-4cd8-b723-d96e63ed7bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:28.000753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911503c3-920d-4cd8-b723-d96e63ed7bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.001039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911503c3-920d-4cd8-b723-d96e63ed7bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.001316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911503c3-920d-4cd8-b723-d96e63ed7bf2 admin admin] PATH_INFO: `/v3/users/311b0a8b22174a278793537963b5d142` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.002456 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911503c3-920d-4cd8-b723-d96e63ed7bf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=311b0a8b22174a278793537963b5d142)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:28.004433 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911503c3-920d-4cd8-b723-d96e63ed7bf2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:28.011719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495b106-7435-4928-a086-c09280c1ca42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:28.012362 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495b106-7435-4928-a086-c09280c1ca42 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.012636 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495b106-7435-4928-a086-c09280c1ca42 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.012905 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495b106-7435-4928-a086-c09280c1ca42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.014626 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2495b106-7435-4928-a086-c09280c1ca42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:28.026443 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495b106-7435-4928-a086-c09280c1ca42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:28.028089 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495b106-7435-4928-a086-c09280c1ca42 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:28.039922 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1407/2832] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:44:28.070273 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-911503c3-920d-4cd8-b723-d96e63ed7bf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '311b0a8b22174a278793537963b5d142'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:28.070952 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-911503c3-920d-4cd8-b723-d96e63ed7bf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '311b0a8b22174a278793537963b5d142'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:28.072225 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-911503c3-920d-4cd8-b723-d96e63ed7bf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '311b0a8b22174a278793537963b5d142'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:28.080822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-911503c3-920d-4cd8-b723-d96e63ed7bf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '311b0a8b22174a278793537963b5d142'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:28.088391 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-911503c3-920d-4cd8-b723-d96e63ed7bf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '311b0a8b22174a278793537963b5d142'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:28.115097 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1426/2833] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:44:27 2022] DELETE /identity/v3/users/311b0a8b22174a278793537963b5d142 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:28.292390 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495b106-7435-4928-a086-c09280c1ca42 service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:28.303614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495b106-7435-4928-a086-c09280c1ca42 service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:28.360970 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:28.361592 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.361871 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.362143 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] PATH_INFO: `/v3/projects/9569b373f21841e8883b0b269073121a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.363238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9569b373f21841e8883b0b269073121a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:28.365167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:28.401781 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:28.413782 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9569b373f21841e8883b0b269073121a', 'request_id': 'req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:28.420493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9569b373f21841e8883b0b269073121a', 'request_id': 'req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:28.426120 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9569b373f21841e8883b0b269073121a', 'request_id': 'req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:28.441265 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] The token cache is being invalidate because project 9569b373f21841e8883b0b269073121a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:44:28.441572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:28.442210 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1bca113a-f8d0-41ae-99dd-0c00e2fde1a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:28.443717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1408/2834] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:44:28 2022] DELETE /identity/v3/projects/9569b373f21841e8883b0b269073121a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:28.448596 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9dc88-e75b-4d77-85b6-870d5b9ec378 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.448907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9dc88-e75b-4d77-85b6-870d5b9ec378 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.449181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9dc88-e75b-4d77-85b6-870d5b9ec378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.470141 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-86d9dc88-e75b-4d77-85b6-870d5b9ec378 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:28.471392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-86d9dc88-e75b-4d77-85b6-870d5b9ec378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:28.531540 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1427/2835] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:28.539527 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a8993-4034-480a-89b2-3cb32eef916f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.539810 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a8993-4034-480a-89b2-3cb32eef916f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.540190 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a8993-4034-480a-89b2-3cb32eef916f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.560746 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-143a8993-4034-480a-89b2-3cb32eef916f None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:28.561963 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-143a8993-4034-480a-89b2-3cb32eef916f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:28.570160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1409/2836] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:28.574403 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9dc88-e75b-4d77-85b6-870d5b9ec378 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:28.584199 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9dc88-e75b-4d77-85b6-870d5b9ec378 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:28.589660 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc5237-a2ff-49ac-9312-874833a4852a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:28.590266 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc5237-a2ff-49ac-9312-874833a4852a admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.590546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc5237-a2ff-49ac-9312-874833a4852a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.590852 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc5237-a2ff-49ac-9312-874833a4852a admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.591509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc5237-a2ff-49ac-9312-874833a4852a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:44:28.591817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc5237-a2ff-49ac-9312-874833a4852a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:28.593551 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc5237-a2ff-49ac-9312-874833a4852a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:28.606985 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1428/2837] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:44:28.611070 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a8993-4034-480a-89b2-3cb32eef916f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:28.621260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a8993-4034-480a-89b2-3cb32eef916f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:28.625944 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f218544-3b78-453f-9ea3-755b2161c4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:28.626531 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f218544-3b78-453f-9ea3-755b2161c4bd admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.626812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f218544-3b78-453f-9ea3-755b2161c4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.627113 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f218544-3b78-453f-9ea3-755b2161c4bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.627788 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f218544-3b78-453f-9ea3-755b2161c4bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:28.629722 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f218544-3b78-453f-9ea3-755b2161c4bd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:28.658844 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1410/2838] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:44:28 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:28.663017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc5237-a2ff-49ac-9312-874833a4852a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:28.673733 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc5237-a2ff-49ac-9312-874833a4852a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:28.677231 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:28.731264 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84633cb-b31a-47c6-93b5-8468e3d2f4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:28.731850 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84633cb-b31a-47c6-93b5-8468e3d2f4ab admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.732157 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84633cb-b31a-47c6-93b5-8468e3d2f4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.732428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84633cb-b31a-47c6-93b5-8468e3d2f4ab admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.733056 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84633cb-b31a-47c6-93b5-8468e3d2f4ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:28.734943 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84633cb-b31a-47c6-93b5-8468e3d2f4ab admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:28.756420 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1429/2839] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:44:28 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:44:28.760813 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f218544-3b78-453f-9ea3-755b2161c4bd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:28.771812 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f218544-3b78-453f-9ea3-755b2161c4bd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:28.776827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1450cb-b745-4a76-b11b-392551ec790b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:28.777421 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1450cb-b745-4a76-b11b-392551ec790b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.777701 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1450cb-b745-4a76-b11b-392551ec790b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.777974 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1450cb-b745-4a76-b11b-392551ec790b admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.778617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1450cb-b745-4a76-b11b-392551ec790b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:28.780254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1450cb-b745-4a76-b11b-392551ec790b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:28.791248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1411/2840] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:28.795806 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84633cb-b31a-47c6-93b5-8468e3d2f4ab admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:28.805932 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84633cb-b31a-47c6-93b5-8468e3d2f4ab admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:28.810515 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478970f-3d3a-411f-8df8-81743a153e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:28.811103 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478970f-3d3a-411f-8df8-81743a153e4d admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.811384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478970f-3d3a-411f-8df8-81743a153e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.811652 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478970f-3d3a-411f-8df8-81743a153e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.812337 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478970f-3d3a-411f-8df8-81743a153e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:28.813919 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478970f-3d3a-411f-8df8-81743a153e4d admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:28.824425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1430/2841] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:28.828617 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1450cb-b745-4a76-b11b-392551ec790b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:28.839239 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1450cb-b745-4a76-b11b-392551ec790b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:28.844732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32cce6-a7f9-465b-a6f7-44faa90e81f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:28.845371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32cce6-a7f9-465b-a6f7-44faa90e81f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.845698 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32cce6-a7f9-465b-a6f7-44faa90e81f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.845968 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32cce6-a7f9-465b-a6f7-44faa90e81f1 admin admin] PATH_INFO: `/v3/projects/86adfebf3e9845bf9c7a4e4121448cdc/users/67b6af05c9ab4329b7eeeb262f7b139e/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.860833 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32cce6-a7f9-465b-a6f7-44faa90e81f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86adfebf3e9845bf9c7a4e4121448cdc, user_id=67b6af05c9ab4329b7eeeb262f7b139e, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:28.863444 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32cce6-a7f9-465b-a6f7-44faa90e81f1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:28.879014 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1412/2842] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:28 2022] PUT /identity/v3/projects/86adfebf3e9845bf9c7a4e4121448cdc/users/67b6af05c9ab4329b7eeeb262f7b139e/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:28.884604 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75570267-e4be-49b2-9c1c-b071d9a92878 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:28.884871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75570267-e4be-49b2-9c1c-b071d9a92878 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:28.885148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75570267-e4be-49b2-9c1c-b071d9a92878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:28.904508 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-75570267-e4be-49b2-9c1c-b071d9a92878 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:28.905711 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-75570267-e4be-49b2-9c1c-b071d9a92878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:28.966877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1431/2843] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:28.976764 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32cce6-a7f9-465b-a6f7-44faa90e81f1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:28.979465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2e32cce6-a7f9-465b-a6f7-44faa90e81f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:29.037743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32cce6-a7f9-465b-a6f7-44faa90e81f1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:29.041570 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcc76c-48d7-43b3-9d91-05b06d0c287a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:29.042167 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcc76c-48d7-43b3-9d91-05b06d0c287a service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:29.042446 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcc76c-48d7-43b3-9d91-05b06d0c287a service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:29.042750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcc76c-48d7-43b3-9d91-05b06d0c287a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:29.052160 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcc76c-48d7-43b3-9d91-05b06d0c287a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:29.053580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcc76c-48d7-43b3-9d91-05b06d0c287a service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:29.062831 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1413/2844] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:44:29.663835 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd4f37-3a48-490a-be68-cd739723d138 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:29.664146 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd4f37-3a48-490a-be68-cd739723d138 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:29.664496 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd4f37-3a48-490a-be68-cd739723d138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:29.694113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d9dd4f37-3a48-490a-be68-cd739723d138 None None] MFA Rules not processed for user `67b6af05c9ab4329b7eeeb262f7b139e`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:29.695398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d9dd4f37-3a48-490a-be68-cd739723d138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:29.762371 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1432/2845] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:29 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:29.767130 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcc76c-48d7-43b3-9d91-05b06d0c287a service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:29.777747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcc76c-48d7-43b3-9d91-05b06d0c287a service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:29.782434 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d66193-0ec0-4778-8762-7d2a8500fa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:29.783035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66193-0ec0-4778-8762-7d2a8500fa3b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:29.783312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66193-0ec0-4778-8762-7d2a8500fa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:29.783580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66193-0ec0-4778-8762-7d2a8500fa3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:29.784224 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d66193-0ec0-4778-8762-7d2a8500fa3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:29.786117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d66193-0ec0-4778-8762-7d2a8500fa3b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:29.816352 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1414/2846] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:44:29 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:29.820424 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd4f37-3a48-490a-be68-cd739723d138 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:29.831493 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd4f37-3a48-490a-be68-cd739723d138 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:29.843019 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:29.890897 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030d923-fef4-4bca-914b-3248d1bb085f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:29.891501 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030d923-fef4-4bca-914b-3248d1bb085f admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:29.891785 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030d923-fef4-4bca-914b-3248d1bb085f admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:29.892086 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030d923-fef4-4bca-914b-3248d1bb085f admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:29.892724 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030d923-fef4-4bca-914b-3248d1bb085f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:29.894649 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030d923-fef4-4bca-914b-3248d1bb085f admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:29.916537 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1433/2847] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:44:29 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:29.920750 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d66193-0ec0-4778-8762-7d2a8500fa3b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:29.931227 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d66193-0ec0-4778-8762-7d2a8500fa3b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:29.935849 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d19565-53d2-4ab0-ba74-2256510e69d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:29.936485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19565-53d2-4ab0-ba74-2256510e69d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:29.936768 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19565-53d2-4ab0-ba74-2256510e69d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:29.937045 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19565-53d2-4ab0-ba74-2256510e69d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:29.937685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d19565-53d2-4ab0-ba74-2256510e69d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:29.939305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d19565-53d2-4ab0-ba74-2256510e69d7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:29.950504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1415/2848] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:29.954545 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030d923-fef4-4bca-914b-3248d1bb085f admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:29.965254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030d923-fef4-4bca-914b-3248d1bb085f admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:29.969899 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2439c-9240-44fd-ab3a-1ee56b5fd70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:29.970544 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2439c-9240-44fd-ab3a-1ee56b5fd70c admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:29.970827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2439c-9240-44fd-ab3a-1ee56b5fd70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:29.971105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2439c-9240-44fd-ab3a-1ee56b5fd70c admin admin] PATH_INFO: `/v3/projects/14c0e6ae6a45410e9274e452b34d859c/users/c73192c7943d47c4bc233aa511e4f213/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:29.986907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2439c-9240-44fd-ab3a-1ee56b5fd70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c0e6ae6a45410e9274e452b34d859c, user_id=c73192c7943d47c4bc233aa511e4f213, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:29.989609 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2439c-9240-44fd-ab3a-1ee56b5fd70c admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:30.002825 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1434/2849] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:29 2022] PUT /identity/v3/projects/14c0e6ae6a45410e9274e452b34d859c/users/c73192c7943d47c4bc233aa511e4f213/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:30.006853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d19565-53d2-4ab0-ba74-2256510e69d7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:30.017553 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d19565-53d2-4ab0-ba74-2256510e69d7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:30.043452 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:30.076299 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16917e40-f260-43be-a91d-9ebe82c53d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:30.076808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16917e40-f260-43be-a91d-9ebe82c53d88 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:30.077088 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16917e40-f260-43be-a91d-9ebe82c53d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:30.077367 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16917e40-f260-43be-a91d-9ebe82c53d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:30.078006 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16917e40-f260-43be-a91d-9ebe82c53d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:30.079590 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16917e40-f260-43be-a91d-9ebe82c53d88 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:30.089634 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1416/2850] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:30.093939 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2439c-9240-44fd-ab3a-1ee56b5fd70c admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:30.104471 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2439c-9240-44fd-ab3a-1ee56b5fd70c admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:30.109018 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc6657-4399-4bee-a6d1-b23876659c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:30.109666 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc6657-4399-4bee-a6d1-b23876659c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:30.109947 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc6657-4399-4bee-a6d1-b23876659c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:30.110220 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc6657-4399-4bee-a6d1-b23876659c79 admin admin] PATH_INFO: `/v3/projects/14c0e6ae6a45410e9274e452b34d859c/users/c73192c7943d47c4bc233aa511e4f213/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:30.112183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc6657-4399-4bee-a6d1-b23876659c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c0e6ae6a45410e9274e452b34d859c, user_id=c73192c7943d47c4bc233aa511e4f213, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:30.114837 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc6657-4399-4bee-a6d1-b23876659c79 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:30.125674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1435/2851] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:30 2022] PUT /identity/v3/projects/14c0e6ae6a45410e9274e452b34d859c/users/c73192c7943d47c4bc233aa511e4f213/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:30.755340 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288766b-a335-4167-9acd-e5671616564d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:30.756145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288766b-a335-4167-9acd-e5671616564d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:30.756145 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288766b-a335-4167-9acd-e5671616564d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:30.790117 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c288766b-a335-4167-9acd-e5671616564d None None] MFA Rules not processed for user `c73192c7943d47c4bc233aa511e4f213`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:30.791868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c288766b-a335-4167-9acd-e5671616564d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:30.863907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1417/2852] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:30 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:31.810147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc6657-4399-4bee-a6d1-b23876659c79 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:31.811951 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-a5bc6657-4399-4bee-a6d1-b23876659c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:31.878568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288766b-a335-4167-9acd-e5671616564d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:31.893346 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc6657-4399-4bee-a6d1-b23876659c79 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:31.898657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450f4fd-4d03-41c3-97b9-bff965709cc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:31.899269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450f4fd-4d03-41c3-97b9-bff965709cc0 service placement] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:31.899598 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450f4fd-4d03-41c3-97b9-bff965709cc0 service placement] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:31.899869 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450f4fd-4d03-41c3-97b9-bff965709cc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:31.901742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c450f4fd-4d03-41c3-97b9-bff965709cc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:31.914272 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450f4fd-4d03-41c3-97b9-bff965709cc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:31.916624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450f4fd-4d03-41c3-97b9-bff965709cc0 service placement] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:31.927857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1436/2853] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:44:31.960929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288766b-a335-4167-9acd-e5671616564d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:31.965486 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:31.966123 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:31.966404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:31.966681 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] PATH_INFO: `/v3/users/67b6af05c9ab4329b7eeeb262f7b139e` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:31.967835 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67b6af05c9ab4329b7eeeb262f7b139e)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:31.969819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:32.024993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67b6af05c9ab4329b7eeeb262f7b139e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.025572 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67b6af05c9ab4329b7eeeb262f7b139e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.026179 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67b6af05c9ab4329b7eeeb262f7b139e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.030033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:32.035802 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67b6af05c9ab4329b7eeeb262f7b139e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.046217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67b6af05c9ab4329b7eeeb262f7b139e'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.065324 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1418/2854] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:44:31 2022] DELETE /identity/v3/users/67b6af05c9ab4329b7eeeb262f7b139e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:32.254432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450f4fd-4d03-41c3-97b9-bff965709cc0 service placement] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:32.265444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450f4fd-4d03-41c3-97b9-bff965709cc0 service placement] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:32.317051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:32.324177 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:32.324964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:32.325300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:32.325809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] PATH_INFO: `/v3/projects/86adfebf3e9845bf9c7a4e4121448cdc` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:32.327126 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86adfebf3e9845bf9c7a4e4121448cdc)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:32.329082 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:32.375329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86adfebf3e9845bf9c7a4e4121448cdc', 'request_id': 'req-c4cef21b-4af5-4281-8500-05ac2b09b562'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.381359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86adfebf3e9845bf9c7a4e4121448cdc', 'request_id': 'req-c4cef21b-4af5-4281-8500-05ac2b09b562'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.387008 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86adfebf3e9845bf9c7a4e4121448cdc', 'request_id': 'req-c4cef21b-4af5-4281-8500-05ac2b09b562'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.399319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] The token cache is being invalidate because project 86adfebf3e9845bf9c7a4e4121448cdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:44:32.400218 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.400857 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.402323 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1437/2855] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:44:32 2022] DELETE /identity/v3/projects/86adfebf3e9845bf9c7a4e4121448cdc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:32.406437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:32.407852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:32.421537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c7d74-4030-41f2-a01b-c74d599135b4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:32.479316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d23b8-51d5-4c96-ac34-82be120d6d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:32.479940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d23b8-51d5-4c96-ac34-82be120d6d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:32.480254 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d23b8-51d5-4c96-ac34-82be120d6d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:32.480528 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d23b8-51d5-4c96-ac34-82be120d6d5e admin admin] PATH_INFO: `/v3/users/c73192c7943d47c4bc233aa511e4f213` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:32.481656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d23b8-51d5-4c96-ac34-82be120d6d5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c73192c7943d47c4bc233aa511e4f213)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:32.483576 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d23b8-51d5-4c96-ac34-82be120d6d5e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:32.532172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1c1d23b8-51d5-4c96-ac34-82be120d6d5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c73192c7943d47c4bc233aa511e4f213'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.532771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1c1d23b8-51d5-4c96-ac34-82be120d6d5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c73192c7943d47c4bc233aa511e4f213'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.533375 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1c1d23b8-51d5-4c96-ac34-82be120d6d5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c73192c7943d47c4bc233aa511e4f213'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.540810 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1c1d23b8-51d5-4c96-ac34-82be120d6d5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c73192c7943d47c4bc233aa511e4f213'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.547988 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1c1d23b8-51d5-4c96-ac34-82be120d6d5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c73192c7943d47c4bc233aa511e4f213'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.565199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1419/2856] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:44:32 2022] DELETE /identity/v3/users/c73192c7943d47c4bc233aa511e4f213 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:32.734775 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:32.736243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:32.749867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cef21b-4af5-4281-8500-05ac2b09b562 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:32.809697 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:32.810246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:32.810791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:32.810791 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] PATH_INFO: `/v3/projects/14c0e6ae6a45410e9274e452b34d859c` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:32.811903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14c0e6ae6a45410e9274e452b34d859c)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:32.813849 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:32.860230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14c0e6ae6a45410e9274e452b34d859c', 'request_id': 'req-c6823300-b9fd-4be1-8b5b-310be5a97ea0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.866427 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14c0e6ae6a45410e9274e452b34d859c', 'request_id': 'req-c6823300-b9fd-4be1-8b5b-310be5a97ea0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.872094 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14c0e6ae6a45410e9274e452b34d859c', 'request_id': 'req-c6823300-b9fd-4be1-8b5b-310be5a97ea0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.885338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] The token cache is being invalidate because project 14c0e6ae6a45410e9274e452b34d859c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:44:32.885624 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.886244 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-c6823300-b9fd-4be1-8b5b-310be5a97ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:32.887754 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1438/2857] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:44:32 2022] DELETE /identity/v3/projects/14c0e6ae6a45410e9274e452b34d859c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:32.892941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5aa45-652b-4587-ab22-c3cc74180b1e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:32.893411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5aa45-652b-4587-ab22-c3cc74180b1e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:32.893502 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5aa45-652b-4587-ab22-c3cc74180b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:32.913211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-09b5aa45-652b-4587-ab22-c3cc74180b1e None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:32.914414 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-09b5aa45-652b-4587-ab22-c3cc74180b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:32.959679 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:32.976909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1420/2858] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:32.984528 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79c6bd-5e54-40ee-bbdf-a661018e6470 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:32.984815 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79c6bd-5e54-40ee-bbdf-a661018e6470 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:32.985090 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79c6bd-5e54-40ee-bbdf-a661018e6470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.005461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-dd79c6bd-5e54-40ee-bbdf-a661018e6470 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:33.006668 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-dd79c6bd-5e54-40ee-bbdf-a661018e6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:33.016387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1439/2859] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:33.020662 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5aa45-652b-4587-ab22-c3cc74180b1e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:33.030792 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5aa45-652b-4587-ab22-c3cc74180b1e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:33.035260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3aa0e-807a-42d3-9a8a-05992df8db13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:33.035841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3aa0e-807a-42d3-9a8a-05992df8db13 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:33.036155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3aa0e-807a-42d3-9a8a-05992df8db13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:33.036425 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3aa0e-807a-42d3-9a8a-05992df8db13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.037061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3aa0e-807a-42d3-9a8a-05992df8db13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:44:33.037359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3aa0e-807a-42d3-9a8a-05992df8db13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:33.038941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3aa0e-807a-42d3-9a8a-05992df8db13 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:33.050604 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1421/2860] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:44:33.055101 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79c6bd-5e54-40ee-bbdf-a661018e6470 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:33.065147 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79c6bd-5e54-40ee-bbdf-a661018e6470 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:33.069979 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed539d-9fd2-4fb2-9527-8c168397ffcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:33.070581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed539d-9fd2-4fb2-9527-8c168397ffcd admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:33.070861 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed539d-9fd2-4fb2-9527-8c168397ffcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:33.071134 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed539d-9fd2-4fb2-9527-8c168397ffcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.072699 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed539d-9fd2-4fb2-9527-8c168397ffcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:33.074382 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed539d-9fd2-4fb2-9527-8c168397ffcd admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:33.084233 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:33.107676 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1440/2861] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:44:33 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:33.108820 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3aa0e-807a-42d3-9a8a-05992df8db13 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:33.119331 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3aa0e-807a-42d3-9a8a-05992df8db13 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:33.178510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d867ed-7f27-45f2-83cc-c9d79bbb23a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:33.179119 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d867ed-7f27-45f2-83cc-c9d79bbb23a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:33.179588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d867ed-7f27-45f2-83cc-c9d79bbb23a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:33.179731 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d867ed-7f27-45f2-83cc-c9d79bbb23a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.180356 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d867ed-7f27-45f2-83cc-c9d79bbb23a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:33.182784 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d867ed-7f27-45f2-83cc-c9d79bbb23a9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:33.204530 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1422/2862] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:44:33 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:44:33.208755 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed539d-9fd2-4fb2-9527-8c168397ffcd admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:33.219247 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed539d-9fd2-4fb2-9527-8c168397ffcd admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:33.223808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9401c97-2dc2-4101-8b92-ca62a4b8bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:33.224432 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9401c97-2dc2-4101-8b92-ca62a4b8bab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:33.224713 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9401c97-2dc2-4101-8b92-ca62a4b8bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:33.224984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9401c97-2dc2-4101-8b92-ca62a4b8bab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.225627 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9401c97-2dc2-4101-8b92-ca62a4b8bab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:33.227704 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9401c97-2dc2-4101-8b92-ca62a4b8bab2 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:33.237776 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1441/2863] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:33.242002 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d867ed-7f27-45f2-83cc-c9d79bbb23a9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:33.253150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d867ed-7f27-45f2-83cc-c9d79bbb23a9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:33.258141 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87386a1-2599-45b5-b659-db20e176f860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:33.259383 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87386a1-2599-45b5-b659-db20e176f860 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:33.259677 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87386a1-2599-45b5-b659-db20e176f860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:33.259951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87386a1-2599-45b5-b659-db20e176f860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.260620 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87386a1-2599-45b5-b659-db20e176f860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:33.262217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87386a1-2599-45b5-b659-db20e176f860 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:33.272898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1423/2864] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:33.277236 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9401c97-2dc2-4101-8b92-ca62a4b8bab2 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:33.287520 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9401c97-2dc2-4101-8b92-ca62a4b8bab2 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:33.292424 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade0877-4951-4e13-a176-8aa05bf5c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:33.293358 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0877-4951-4e13-a176-8aa05bf5c296 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:33.293635 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0877-4951-4e13-a176-8aa05bf5c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:33.293907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0877-4951-4e13-a176-8aa05bf5c296 admin admin] PATH_INFO: `/v3/projects/79238700f1c3429182e013fb74f62561/users/bf09c48838244c3bbd67f22c1ec9f982/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.308855 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade0877-4951-4e13-a176-8aa05bf5c296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79238700f1c3429182e013fb74f62561, user_id=bf09c48838244c3bbd67f22c1ec9f982, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:33.311500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade0877-4951-4e13-a176-8aa05bf5c296 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:33.324867 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1442/2865] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:33 2022] PUT /identity/v3/projects/79238700f1c3429182e013fb74f62561/users/bf09c48838244c3bbd67f22c1ec9f982/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:33.335964 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c37d6c-18ac-4b35-91d1-b4458be2f1e4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:33.336273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c37d6c-18ac-4b35-91d1-b4458be2f1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:33.336559 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c37d6c-18ac-4b35-91d1-b4458be2f1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.363231 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d2c37d6c-18ac-4b35-91d1-b4458be2f1e4 None None] MFA Rules not processed for user `bf09c48838244c3bbd67f22c1ec9f982`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:33.364467 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d2c37d6c-18ac-4b35-91d1-b4458be2f1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:33.428768 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1424/2866] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:33.440538 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade0877-4951-4e13-a176-8aa05bf5c296 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:33.442230 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-0ade0877-4951-4e13-a176-8aa05bf5c296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:33.498052 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade0877-4951-4e13-a176-8aa05bf5c296 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:33.501686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bb3e1-3e70-4d62-9bdf-0e1e816c3e41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:33.502276 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bb3e1-3e70-4d62-9bdf-0e1e816c3e41 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:33.502556 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bb3e1-3e70-4d62-9bdf-0e1e816c3e41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:33.502829 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bb3e1-3e70-4d62-9bdf-0e1e816c3e41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.525095 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bb3e1-3e70-4d62-9bdf-0e1e816c3e41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:33.526550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bb3e1-3e70-4d62-9bdf-0e1e816c3e41 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:33.539450 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1443/2867] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:44:33.561862 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c37d6c-18ac-4b35-91d1-b4458be2f1e4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:33.606379 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:33.624911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c37d6c-18ac-4b35-91d1-b4458be2f1e4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:33.629542 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:33.630172 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:33.630448 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:33.630773 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] PATH_INFO: `/v3/users/bf09c48838244c3bbd67f22c1ec9f982` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.631971 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf09c48838244c3bbd67f22c1ec9f982)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:33.634034 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:33.679040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf09c48838244c3bbd67f22c1ec9f982'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:33.679678 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf09c48838244c3bbd67f22c1ec9f982'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:33.680307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf09c48838244c3bbd67f22c1ec9f982'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:33.687983 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf09c48838244c3bbd67f22c1ec9f982'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:33.694804 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf09c48838244c3bbd67f22c1ec9f982'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:33.711504 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1425/2868] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:44:33 2022] DELETE /identity/v3/users/bf09c48838244c3bbd67f22c1ec9f982 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:33.716131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2eb2-8299-4460-b076-27cec66b4db8 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:33.716414 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2eb2-8299-4460-b076-27cec66b4db8 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:33.716686 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2eb2-8299-4460-b076-27cec66b4db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.735577 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-318c2eb2-8299-4460-b076-27cec66b4db8 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:33.736809 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-318c2eb2-8299-4460-b076-27cec66b4db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:33.796161 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1444/2869] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:33.805719 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:33.807353 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:33.863720 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b2a74-71a2-4929-b9c4-155e859dfe22 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:33.867316 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332640d-1762-4f65-9a12-4c4470957f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:33.867920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332640d-1762-4f65-9a12-4c4470957f8d service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:33.868262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332640d-1762-4f65-9a12-4c4470957f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:33.868541 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332640d-1762-4f65-9a12-4c4470957f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:33.877488 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332640d-1762-4f65-9a12-4c4470957f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:33.878915 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332640d-1762-4f65-9a12-4c4470957f8d service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:33.888319 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1426/2870] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:44:34.158793 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c2eb2-8299-4460-b076-27cec66b4db8 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:34.160446 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-318c2eb2-8299-4460-b076-27cec66b4db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:34.173384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c2eb2-8299-4460-b076-27cec66b4db8 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:34.181328 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:34.182192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.182743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.183024 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] PATH_INFO: `/v3/projects/79238700f1c3429182e013fb74f62561` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.184368 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79238700f1c3429182e013fb74f62561)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:34.186367 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:34.236032 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79238700f1c3429182e013fb74f62561', 'request_id': 'req-462df9bd-c05a-4874-be71-72a0eddde9d0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:34.237920 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:34.242665 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79238700f1c3429182e013fb74f62561', 'request_id': 'req-462df9bd-c05a-4874-be71-72a0eddde9d0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:34.249404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79238700f1c3429182e013fb74f62561', 'request_id': 'req-462df9bd-c05a-4874-be71-72a0eddde9d0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:34.261893 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] The token cache is being invalidate because project 79238700f1c3429182e013fb74f62561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:44:34.262183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:34.262816 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-462df9bd-c05a-4874-be71-72a0eddde9d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:34.264351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1445/2871] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:44:34 2022] DELETE /identity/v3/projects/79238700f1c3429182e013fb74f62561 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:34.269244 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4924ce8-6f8c-4782-a1fb-9d6c72f51e46 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.269712 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4924ce8-6f8c-4782-a1fb-9d6c72f51e46 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.269834 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4924ce8-6f8c-4782-a1fb-9d6c72f51e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.289940 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b4924ce8-6f8c-4782-a1fb-9d6c72f51e46 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:34.291164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b4924ce8-6f8c-4782-a1fb-9d6c72f51e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:34.349199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1427/2872] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:34.356749 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff95e5-de31-4c33-8464-8648159bf480 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.357041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff95e5-de31-4c33-8464-8648159bf480 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.357317 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff95e5-de31-4c33-8464-8648159bf480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.376927 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-ddff95e5-de31-4c33-8464-8648159bf480 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:34.378128 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ddff95e5-de31-4c33-8464-8648159bf480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:34.387005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1446/2873] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:34.391247 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4924ce8-6f8c-4782-a1fb-9d6c72f51e46 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:34.401496 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4924ce8-6f8c-4782-a1fb-9d6c72f51e46 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:34.405930 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f996b-33de-4218-bf7d-19315c338dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:34.406519 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f996b-33de-4218-bf7d-19315c338dfa admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.406797 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f996b-33de-4218-bf7d-19315c338dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.407069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f996b-33de-4218-bf7d-19315c338dfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.407707 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f996b-33de-4218-bf7d-19315c338dfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:44:34.408008 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f996b-33de-4218-bf7d-19315c338dfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:34.409642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f996b-33de-4218-bf7d-19315c338dfa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:34.411067 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:34.423398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1428/2874] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:44:34.427648 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff95e5-de31-4c33-8464-8648159bf480 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:34.436961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff95e5-de31-4c33-8464-8648159bf480 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:34.441658 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d15607-23cf-477b-804c-fd9572ed1b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:34.442252 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d15607-23cf-477b-804c-fd9572ed1b2b admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.442532 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d15607-23cf-477b-804c-fd9572ed1b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.442804 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d15607-23cf-477b-804c-fd9572ed1b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.443450 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d15607-23cf-477b-804c-fd9572ed1b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:34.445387 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d15607-23cf-477b-804c-fd9572ed1b2b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:34.473570 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1447/2875] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:44:34 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:34.477732 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f996b-33de-4218-bf7d-19315c338dfa admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:34.487788 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f996b-33de-4218-bf7d-19315c338dfa admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:34.544693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb64a-5bd7-47d1-8181-db4bf45b814b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:34.545288 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb64a-5bd7-47d1-8181-db4bf45b814b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.545569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb64a-5bd7-47d1-8181-db4bf45b814b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.545841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb64a-5bd7-47d1-8181-db4bf45b814b admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.546461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cb64a-5bd7-47d1-8181-db4bf45b814b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:34.548407 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cb64a-5bd7-47d1-8181-db4bf45b814b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:34.569342 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1429/2876] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:44:34 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:34.573989 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d15607-23cf-477b-804c-fd9572ed1b2b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:34.583983 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d15607-23cf-477b-804c-fd9572ed1b2b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:34.588827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db770146-5ad6-4d80-8e26-88f188343409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:34.589423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db770146-5ad6-4d80-8e26-88f188343409 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.589702 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db770146-5ad6-4d80-8e26-88f188343409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.589974 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db770146-5ad6-4d80-8e26-88f188343409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.590613 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db770146-5ad6-4d80-8e26-88f188343409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:34.592243 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db770146-5ad6-4d80-8e26-88f188343409 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:34.602118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1448/2877] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:34.606021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb64a-5bd7-47d1-8181-db4bf45b814b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:34.615684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb64a-5bd7-47d1-8181-db4bf45b814b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:34.620841 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaed69e-cfbc-439f-9980-ed4196e6ee21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:34.621423 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaed69e-cfbc-439f-9980-ed4196e6ee21 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.621705 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaed69e-cfbc-439f-9980-ed4196e6ee21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.621979 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaed69e-cfbc-439f-9980-ed4196e6ee21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.622643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaed69e-cfbc-439f-9980-ed4196e6ee21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:34.624258 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaed69e-cfbc-439f-9980-ed4196e6ee21 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:34.635134 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1430/2878] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:44:34.639049 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db770146-5ad6-4d80-8e26-88f188343409 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:34.649064 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db770146-5ad6-4d80-8e26-88f188343409 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:34.653789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072a247-6090-4b0f-8eb8-30c49a2e85d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:34.654359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a247-6090-4b0f-8eb8-30c49a2e85d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.654641 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a247-6090-4b0f-8eb8-30c49a2e85d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.654912 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a247-6090-4b0f-8eb8-30c49a2e85d9 admin admin] PATH_INFO: `/v3/projects/45044e8b614d49159001caf30ce81000/users/3dbeec1a367c4bfd963b81505be90c15/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.669723 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072a247-6090-4b0f-8eb8-30c49a2e85d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45044e8b614d49159001caf30ce81000, user_id=3dbeec1a367c4bfd963b81505be90c15, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:34.672455 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072a247-6090-4b0f-8eb8-30c49a2e85d9 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:34.685426 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1449/2879] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:34 2022] PUT /identity/v3/projects/45044e8b614d49159001caf30ce81000/users/3dbeec1a367c4bfd963b81505be90c15/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:34.695588 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828aa1-e813-434e-b32e-7e08454d8a17 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.695870 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828aa1-e813-434e-b32e-7e08454d8a17 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.696848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828aa1-e813-434e-b32e-7e08454d8a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.725411 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-42828aa1-e813-434e-b32e-7e08454d8a17 None None] MFA Rules not processed for user `3dbeec1a367c4bfd963b81505be90c15`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:34.726664 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-42828aa1-e813-434e-b32e-7e08454d8a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:34.791924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1431/2880] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:44:34.796655 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072a247-6090-4b0f-8eb8-30c49a2e85d9 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:34.860062 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072a247-6090-4b0f-8eb8-30c49a2e85d9 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:34.864764 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07495b84-7e38-4f70-b66d-dd027ffc9e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:34.865383 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07495b84-7e38-4f70-b66d-dd027ffc9e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.865661 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07495b84-7e38-4f70-b66d-dd027ffc9e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.865963 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07495b84-7e38-4f70-b66d-dd027ffc9e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.866578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07495b84-7e38-4f70-b66d-dd027ffc9e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:34.869002 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07495b84-7e38-4f70-b66d-dd027ffc9e77 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:34.898094 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1450/2881] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:44:34 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:34.901829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42828aa1-e813-434e-b32e-7e08454d8a17 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:34.914361 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42828aa1-e813-434e-b32e-7e08454d8a17 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:34.971105 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7b18d-1a8e-4316-aa1a-22e9e60bc9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:34.971708 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7b18d-1a8e-4316-aa1a-22e9e60bc9dd admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:34.971996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7b18d-1a8e-4316-aa1a-22e9e60bc9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:34.972298 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7b18d-1a8e-4316-aa1a-22e9e60bc9dd admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:34.972972 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7b18d-1a8e-4316-aa1a-22e9e60bc9dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:34.974838 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7b18d-1a8e-4316-aa1a-22e9e60bc9dd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:34.983973 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:34.996803 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1432/2882] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:44:34 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:44:35.000926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07495b84-7e38-4f70-b66d-dd027ffc9e77 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:35.011241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07495b84-7e38-4f70-b66d-dd027ffc9e77 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:35.015871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce824e-2d93-40ec-960b-4948fc2f2d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:35.016500 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce824e-2d93-40ec-960b-4948fc2f2d3b admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:35.016827 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce824e-2d93-40ec-960b-4948fc2f2d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:35.017105 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce824e-2d93-40ec-960b-4948fc2f2d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:35.017753 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce824e-2d93-40ec-960b-4948fc2f2d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:35.019342 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce824e-2d93-40ec-960b-4948fc2f2d3b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:35.030044 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1451/2883] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:35.034458 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7b18d-1a8e-4316-aa1a-22e9e60bc9dd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:35.045470 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7b18d-1a8e-4316-aa1a-22e9e60bc9dd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:35.050873 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d5e3f-63d6-418c-bbdc-44b86ca9a13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:35.051564 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d5e3f-63d6-418c-bbdc-44b86ca9a13a admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:35.051859 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d5e3f-63d6-418c-bbdc-44b86ca9a13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:35.052155 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d5e3f-63d6-418c-bbdc-44b86ca9a13a admin admin] PATH_INFO: `/v3/projects/98d9873cdb7d4900b3f8c94f06c242f8/users/ec50f99c9e814d28a2c8351d60d29a37/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:35.067736 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d5e3f-63d6-418c-bbdc-44b86ca9a13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d9873cdb7d4900b3f8c94f06c242f8, user_id=ec50f99c9e814d28a2c8351d60d29a37, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:35.070433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d5e3f-63d6-418c-bbdc-44b86ca9a13a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:35.083730 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1433/2884] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:35 2022] PUT /identity/v3/projects/98d9873cdb7d4900b3f8c94f06c242f8/users/ec50f99c9e814d28a2c8351d60d29a37/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:44:35.087903 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce824e-2d93-40ec-960b-4948fc2f2d3b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:35.098051 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce824e-2d93-40ec-960b-4948fc2f2d3b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:35.145181 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:35.157488 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559ee7f-f120-4c19-afe6-e655d7733513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:35.158096 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559ee7f-f120-4c19-afe6-e655d7733513 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:35.158380 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559ee7f-f120-4c19-afe6-e655d7733513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:35.158657 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559ee7f-f120-4c19-afe6-e655d7733513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:35.159300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559ee7f-f120-4c19-afe6-e655d7733513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:35.160988 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559ee7f-f120-4c19-afe6-e655d7733513 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:35.171863 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1452/2885] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:35.176163 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d5e3f-63d6-418c-bbdc-44b86ca9a13a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:35.187035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d5e3f-63d6-418c-bbdc-44b86ca9a13a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:35.191880 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ea44f-65f9-465b-b926-de30d9f899b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:35.192558 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ea44f-65f9-465b-b926-de30d9f899b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:35.192842 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ea44f-65f9-465b-b926-de30d9f899b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:35.193114 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ea44f-65f9-465b-b926-de30d9f899b8 admin admin] PATH_INFO: `/v3/projects/98d9873cdb7d4900b3f8c94f06c242f8/users/ec50f99c9e814d28a2c8351d60d29a37/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:35.195096 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ea44f-65f9-465b-b926-de30d9f899b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d9873cdb7d4900b3f8c94f06c242f8, user_id=ec50f99c9e814d28a2c8351d60d29a37, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:35.197752 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ea44f-65f9-465b-b926-de30d9f899b8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:35.208032 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1434/2886] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:35 2022] PUT /identity/v3/projects/98d9873cdb7d4900b3f8c94f06c242f8/users/ec50f99c9e814d28a2c8351d60d29a37/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:35.219088 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fab8e-d9d4-4ad4-931b-b9b734caecab None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:35.219377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fab8e-d9d4-4ad4-931b-b9b734caecab None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:35.219653 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fab8e-d9d4-4ad4-931b-b9b734caecab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:35.248423 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-cd0fab8e-d9d4-4ad4-931b-b9b734caecab None None] MFA Rules not processed for user `ec50f99c9e814d28a2c8351d60d29a37`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:35.249639 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-cd0fab8e-d9d4-4ad4-931b-b9b734caecab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:35.316807 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1453/2887] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:35.327371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ea44f-65f9-465b-b926-de30d9f899b8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:35.329081 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a12ea44f-65f9-465b-b926-de30d9f899b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:35.385728 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ea44f-65f9-465b-b926-de30d9f899b8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:35.389451 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af108289-7a16-4bd5-baef-b8523d434309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:35.390037 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af108289-7a16-4bd5-baef-b8523d434309 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:35.390312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af108289-7a16-4bd5-baef-b8523d434309 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:35.390580 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af108289-7a16-4bd5-baef-b8523d434309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:35.445404 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af108289-7a16-4bd5-baef-b8523d434309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:35.446823 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af108289-7a16-4bd5-baef-b8523d434309 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:35.455450 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1435/2888] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:44:37.981365 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fab8e-d9d4-4ad4-931b-b9b734caecab None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:38.046352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fab8e-d9d4-4ad4-931b-b9b734caecab None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:38.051907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:38.052565 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:38.052845 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:38.053118 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] PATH_INFO: `/v3/users/3dbeec1a367c4bfd963b81505be90c15` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:38.054289 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dbeec1a367c4bfd963b81505be90c15)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:38.056563 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:38.103357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dbeec1a367c4bfd963b81505be90c15'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.104027 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dbeec1a367c4bfd963b81505be90c15'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.104687 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dbeec1a367c4bfd963b81505be90c15'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.111926 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dbeec1a367c4bfd963b81505be90c15'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.119149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dbeec1a367c4bfd963b81505be90c15'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.126839 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:38.138602 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1454/2889] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:44:37 2022] DELETE /identity/v3/users/3dbeec1a367c4bfd963b81505be90c15 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:44:38.143185 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e5116-ef97-4775-ab43-38782eb7fba6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:38.143643 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e5116-ef97-4775-ab43-38782eb7fba6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:38.143733 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e5116-ef97-4775-ab43-38782eb7fba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:38.163001 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-631e5116-ef97-4775-ab43-38782eb7fba6 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:38.164528 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-631e5116-ef97-4775-ab43-38782eb7fba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:38.222514 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1436/2890] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:38.232329 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:38.233846 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:38.290319 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49630f65-99da-44ee-8852-2bcd249035be admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:38.294400 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee31bc6-a288-4883-bdfc-b24410902523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:38.294997 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee31bc6-a288-4883-bdfc-b24410902523 service neutron] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:38.295275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee31bc6-a288-4883-bdfc-b24410902523 service neutron] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:38.295546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee31bc6-a288-4883-bdfc-b24410902523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:38.304404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee31bc6-a288-4883-bdfc-b24410902523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:38.305744 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee31bc6-a288-4883-bdfc-b24410902523 service neutron] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:38.315882 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1455/2891] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:44:38.579238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e5116-ef97-4775-ab43-38782eb7fba6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:38.580726 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-631e5116-ef97-4775-ab43-38782eb7fba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:38.593849 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e5116-ef97-4775-ab43-38782eb7fba6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:38.598460 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:38.599112 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:38.599389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:38.599659 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] PATH_INFO: `/v3/projects/45044e8b614d49159001caf30ce81000` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:38.600787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] RBAC: Authorizing `identity:delete_project(project_id=45044e8b614d49159001caf30ce81000)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:38.602879 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:38.606054 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:38.651468 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45044e8b614d49159001caf30ce81000', 'request_id': 'req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.657658 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45044e8b614d49159001caf30ce81000', 'request_id': 'req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.663042 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45044e8b614d49159001caf30ce81000', 'request_id': 'req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.675951 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] The token cache is being invalidate because project 45044e8b614d49159001caf30ce81000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:44:38.676321 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.676941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.678405 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1437/2892] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:44:38 2022] DELETE /identity/v3/projects/45044e8b614d49159001caf30ce81000 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:38.683121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee31bc6-a288-4883-bdfc-b24410902523 service neutron] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:38.692988 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee31bc6-a288-4883-bdfc-b24410902523 service neutron] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:38.747905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e238d11-f011-4f91-a3dd-a45ef7858891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:38.748559 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e238d11-f011-4f91-a3dd-a45ef7858891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:38.748833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e238d11-f011-4f91-a3dd-a45ef7858891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:38.749102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e238d11-f011-4f91-a3dd-a45ef7858891 admin admin] PATH_INFO: `/v3/users/ec50f99c9e814d28a2c8351d60d29a37` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:38.750201 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e238d11-f011-4f91-a3dd-a45ef7858891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec50f99c9e814d28a2c8351d60d29a37)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:38.752125 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e238d11-f011-4f91-a3dd-a45ef7858891 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:38.797789 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3e238d11-f011-4f91-a3dd-a45ef7858891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec50f99c9e814d28a2c8351d60d29a37'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.798435 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3e238d11-f011-4f91-a3dd-a45ef7858891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec50f99c9e814d28a2c8351d60d29a37'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.799031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3e238d11-f011-4f91-a3dd-a45ef7858891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec50f99c9e814d28a2c8351d60d29a37'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.807084 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3e238d11-f011-4f91-a3dd-a45ef7858891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec50f99c9e814d28a2c8351d60d29a37'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.814014 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-3e238d11-f011-4f91-a3dd-a45ef7858891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec50f99c9e814d28a2c8351d60d29a37'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:38.830603 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1456/2893] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:44:38 2022] DELETE /identity/v3/users/ec50f99c9e814d28a2c8351d60d29a37 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:39.101575 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:39.103000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:39.116811 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237dec8-2f9b-4fa9-8cb2-75a24fbf908d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:39.174655 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:39.175338 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.175552 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.175861 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] PATH_INFO: `/v3/projects/98d9873cdb7d4900b3f8c94f06c242f8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.177016 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d9873cdb7d4900b3f8c94f06c242f8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:39.178927 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:39.229384 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d9873cdb7d4900b3f8c94f06c242f8', 'request_id': 'req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:39.235413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d9873cdb7d4900b3f8c94f06c242f8', 'request_id': 'req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:39.240848 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d9873cdb7d4900b3f8c94f06c242f8', 'request_id': 'req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:39.253238 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] The token cache is being invalidate because project 98d9873cdb7d4900b3f8c94f06c242f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:44:39.253525 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:39.254139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2bba5e24-0d46-41a6-adb5-e5e2a4409d3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:39.255598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1438/2894] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:44:39 2022] DELETE /identity/v3/projects/98d9873cdb7d4900b3f8c94f06c242f8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:39.260688 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe97fb-4b61-4d82-bd46-dd66532169bf None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.261193 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe97fb-4b61-4d82-bd46-dd66532169bf None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.261282 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe97fb-4b61-4d82-bd46-dd66532169bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.281163 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-2ffe97fb-4b61-4d82-bd46-dd66532169bf None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:39.282376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-2ffe97fb-4b61-4d82-bd46-dd66532169bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:39.290748 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:39.342685 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1457/2895] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:44:39.350800 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9553da8-923a-4d3d-8667-0d6ba4f10750 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.351025 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9553da8-923a-4d3d-8667-0d6ba4f10750 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.351304 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9553da8-923a-4d3d-8667-0d6ba4f10750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.372169 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f9553da8-923a-4d3d-8667-0d6ba4f10750 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:39.373371 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f9553da8-923a-4d3d-8667-0d6ba4f10750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:39.381212 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1439/2896] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:39.385787 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe97fb-4b61-4d82-bd46-dd66532169bf None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:39.395331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe97fb-4b61-4d82-bd46-dd66532169bf None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:39.400149 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c39f9-dad4-4e89-ad2f-fb95d51c1525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:39.401331 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c39f9-dad4-4e89-ad2f-fb95d51c1525 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.401608 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c39f9-dad4-4e89-ad2f-fb95d51c1525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.401877 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c39f9-dad4-4e89-ad2f-fb95d51c1525 admin admin] PATH_INFO: `/v3/domains` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.402518 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c39f9-dad4-4e89-ad2f-fb95d51c1525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:44:39.402817 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c39f9-dad4-4e89-ad2f-fb95d51c1525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:39.404451 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c39f9-dad4-4e89-ad2f-fb95d51c1525 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:39.417190 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1458/2897] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:44:39.421817 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9553da8-923a-4d3d-8667-0d6ba4f10750 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:39.430873 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9553da8-923a-4d3d-8667-0d6ba4f10750 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:39.435600 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4f0a9-8338-4dc1-9002-0c3189323aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:39.436245 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4f0a9-8338-4dc1-9002-0c3189323aea admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.436523 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4f0a9-8338-4dc1-9002-0c3189323aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.436819 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4f0a9-8338-4dc1-9002-0c3189323aea admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.437430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4f0a9-8338-4dc1-9002-0c3189323aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:39.439357 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4f0a9-8338-4dc1-9002-0c3189323aea admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:39.454743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:39.468437 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1440/2898] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:44:39 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:39.472445 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c39f9-dad4-4e89-ad2f-fb95d51c1525 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:39.482357 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c39f9-dad4-4e89-ad2f-fb95d51c1525 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:39.538573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269b036-5aae-4cec-a57c-6299170e440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:39.539166 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b036-5aae-4cec-a57c-6299170e440a admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.539449 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b036-5aae-4cec-a57c-6299170e440a admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.539717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b036-5aae-4cec-a57c-6299170e440a admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.540378 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269b036-5aae-4cec-a57c-6299170e440a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:39.542304 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269b036-5aae-4cec-a57c-6299170e440a admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:39.561988 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1459/2899] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:44:39 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:39.566033 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4f0a9-8338-4dc1-9002-0c3189323aea admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:39.576579 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4f0a9-8338-4dc1-9002-0c3189323aea admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:39.581069 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d77160-af02-4c5e-96ab-1c06dfbdf005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:39.581656 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77160-af02-4c5e-96ab-1c06dfbdf005 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.581933 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77160-af02-4c5e-96ab-1c06dfbdf005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.582204 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77160-af02-4c5e-96ab-1c06dfbdf005 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.582839 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d77160-af02-4c5e-96ab-1c06dfbdf005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:39.584456 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d77160-af02-4c5e-96ab-1c06dfbdf005 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:39.595303 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1441/2900] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:39.599121 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269b036-5aae-4cec-a57c-6299170e440a admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:39.609246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269b036-5aae-4cec-a57c-6299170e440a admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:39.613833 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34afd2-8fe8-4183-8e2a-4369835f1699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:39.614425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34afd2-8fe8-4183-8e2a-4369835f1699 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.614709 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34afd2-8fe8-4183-8e2a-4369835f1699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.614984 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34afd2-8fe8-4183-8e2a-4369835f1699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.615625 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34afd2-8fe8-4183-8e2a-4369835f1699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:39.617254 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34afd2-8fe8-4183-8e2a-4369835f1699 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:39.627743 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1460/2901] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:39.631740 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d77160-af02-4c5e-96ab-1c06dfbdf005 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:39.642709 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d77160-af02-4c5e-96ab-1c06dfbdf005 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:39.647465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48899f3b-1865-47f3-8b75-0f320220c679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:39.648262 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48899f3b-1865-47f3-8b75-0f320220c679 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.648537 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48899f3b-1865-47f3-8b75-0f320220c679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.648807 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48899f3b-1865-47f3-8b75-0f320220c679 admin admin] PATH_INFO: `/v3/projects/556d4306d05e40508eee690bf8c3a7bb/users/c0689d5d4c314f388f029889424b2169/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.664021 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48899f3b-1865-47f3-8b75-0f320220c679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556d4306d05e40508eee690bf8c3a7bb, user_id=c0689d5d4c314f388f029889424b2169, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:39.666885 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48899f3b-1865-47f3-8b75-0f320220c679 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:39.679389 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1442/2902] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:39 2022] PUT /identity/v3/projects/556d4306d05e40508eee690bf8c3a7bb/users/c0689d5d4c314f388f029889424b2169/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:39.689476 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d5afd-6458-48c5-91ff-449bcdabc148 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.689769 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d5afd-6458-48c5-91ff-449bcdabc148 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.690046 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d5afd-6458-48c5-91ff-449bcdabc148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.717933 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-e49d5afd-6458-48c5-91ff-449bcdabc148 None None] MFA Rules not processed for user `c0689d5d4c314f388f029889424b2169`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:39.719192 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-e49d5afd-6458-48c5-91ff-449bcdabc148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:39.783351 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1461/2903] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:39.787827 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48899f3b-1865-47f3-8b75-0f320220c679 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:39.849359 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48899f3b-1865-47f3-8b75-0f320220c679 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:39.854092 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bb45f-ab38-4882-8d8c-2eaa28ed06aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:39.854693 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bb45f-ab38-4882-8d8c-2eaa28ed06aa admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.854976 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bb45f-ab38-4882-8d8c-2eaa28ed06aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.855248 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bb45f-ab38-4882-8d8c-2eaa28ed06aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.855865 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8bb45f-ab38-4882-8d8c-2eaa28ed06aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:39.857795 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8bb45f-ab38-4882-8d8c-2eaa28ed06aa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:39.891314 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1443/2904] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:44:39 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:39.895411 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d5afd-6458-48c5-91ff-449bcdabc148 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:39.906179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d5afd-6458-48c5-91ff-449bcdabc148 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:39.910667 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:39.963209 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee3587-557d-4dfb-9681-e138abce8d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:39.963808 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee3587-557d-4dfb-9681-e138abce8d63 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:39.964113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee3587-557d-4dfb-9681-e138abce8d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:39.964392 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee3587-557d-4dfb-9681-e138abce8d63 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:39.965018 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee3587-557d-4dfb-9681-e138abce8d63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:39.966957 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee3587-557d-4dfb-9681-e138abce8d63 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:39.987510 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1462/2905] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:44:39 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:39.991430 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bb45f-ab38-4882-8d8c-2eaa28ed06aa admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:40.002214 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bb45f-ab38-4882-8d8c-2eaa28ed06aa admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:40.006684 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5d50a-f6b0-4494-ba41-97bf7a87762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:40.007288 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5d50a-f6b0-4494-ba41-97bf7a87762f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.007544 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5d50a-f6b0-4494-ba41-97bf7a87762f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.007815 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5d50a-f6b0-4494-ba41-97bf7a87762f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.008484 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5d50a-f6b0-4494-ba41-97bf7a87762f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:40.010710 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5d50a-f6b0-4494-ba41-97bf7a87762f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:40.021035 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1444/2906] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:40.025039 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee3587-557d-4dfb-9681-e138abce8d63 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:40.035059 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee3587-557d-4dfb-9681-e138abce8d63 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:40.040000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfc988-8a05-4238-86ac-af3762bcd2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:40.040626 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfc988-8a05-4238-86ac-af3762bcd2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.040907 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfc988-8a05-4238-86ac-af3762bcd2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.041182 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfc988-8a05-4238-86ac-af3762bcd2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.041832 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfc988-8a05-4238-86ac-af3762bcd2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:40.043700 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfc988-8a05-4238-86ac-af3762bcd2d6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:40.054182 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1463/2907] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:40.058914 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5d50a-f6b0-4494-ba41-97bf7a87762f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:40.068987 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5d50a-f6b0-4494-ba41-97bf7a87762f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:40.074019 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aef562-a4bf-4864-9014-9a4d5a4e312d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:40.074652 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aef562-a4bf-4864-9014-9a4d5a4e312d admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.074929 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aef562-a4bf-4864-9014-9a4d5a4e312d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.075202 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aef562-a4bf-4864-9014-9a4d5a4e312d admin admin] PATH_INFO: `/v3/projects/d3d3959c58644bfe9da796e3fa2918b2/users/31ce3776832846229b2ca3f0d23c35cf/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.091293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aef562-a4bf-4864-9014-9a4d5a4e312d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d3959c58644bfe9da796e3fa2918b2, user_id=31ce3776832846229b2ca3f0d23c35cf, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:40.093955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aef562-a4bf-4864-9014-9a4d5a4e312d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:40.106455 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1445/2908] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:40 2022] PUT /identity/v3/projects/d3d3959c58644bfe9da796e3fa2918b2/users/31ce3776832846229b2ca3f0d23c35cf/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:40.117800 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797fb6-2cd2-4586-b653-b8eadf0a1e90 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.118023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797fb6-2cd2-4586-b653-b8eadf0a1e90 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.118298 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797fb6-2cd2-4586-b653-b8eadf0a1e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.147063 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-75797fb6-2cd2-4586-b653-b8eadf0a1e90 None None] MFA Rules not processed for user `31ce3776832846229b2ca3f0d23c35cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:40.148477 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-75797fb6-2cd2-4586-b653-b8eadf0a1e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:40.216279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1464/2909] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:40.219996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aef562-a4bf-4864-9014-9a4d5a4e312d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:40.281851 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aef562-a4bf-4864-9014-9a4d5a4e312d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:40.286418 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2af31-bef8-4dab-a19f-fb8c3e9e6729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:40.287011 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2af31-bef8-4dab-a19f-fb8c3e9e6729 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.287286 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2af31-bef8-4dab-a19f-fb8c3e9e6729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.287557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2af31-bef8-4dab-a19f-fb8c3e9e6729 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.288223 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2af31-bef8-4dab-a19f-fb8c3e9e6729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:40.290307 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2af31-bef8-4dab-a19f-fb8c3e9e6729 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:40.304787 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:40.321209 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1446/2910] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:44:40 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:44:40.325281 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797fb6-2cd2-4586-b653-b8eadf0a1e90 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:40.336441 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797fb6-2cd2-4586-b653-b8eadf0a1e90 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:40.392782 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c88ac-2886-407a-9128-bc58bf76c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:40.393301 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c88ac-2886-407a-9128-bc58bf76c296 admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.393581 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c88ac-2886-407a-9128-bc58bf76c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.393851 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c88ac-2886-407a-9128-bc58bf76c296 admin admin] PATH_INFO: `/v3/users` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.394461 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c88ac-2886-407a-9128-bc58bf76c296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:40.396428 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c88ac-2886-407a-9128-bc58bf76c296 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:40.417425 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1465/2911] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:44:40 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:40.421699 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2af31-bef8-4dab-a19f-fb8c3e9e6729 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:40.432347 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2af31-bef8-4dab-a19f-fb8c3e9e6729 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:40.436875 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae2e23-5f95-46f1-b613-3a81ea4e16c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:40.437461 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae2e23-5f95-46f1-b613-3a81ea4e16c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.437747 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae2e23-5f95-46f1-b613-3a81ea4e16c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.438020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae2e23-5f95-46f1-b613-3a81ea4e16c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.438686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae2e23-5f95-46f1-b613-3a81ea4e16c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:40.440312 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae2e23-5f95-46f1-b613-3a81ea4e16c4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:40.450785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1447/2912] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:40.454973 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c88ac-2886-407a-9128-bc58bf76c296 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:40.465470 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c88ac-2886-407a-9128-bc58bf76c296 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:40.470109 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128516bb-3c55-4a59-9356-cb9afb6a3dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:40.470757 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128516bb-3c55-4a59-9356-cb9afb6a3dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.471038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128516bb-3c55-4a59-9356-cb9afb6a3dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.471316 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128516bb-3c55-4a59-9356-cb9afb6a3dbf admin admin] PATH_INFO: `/v3/projects/d233074525a748a9abd2d7786bfcea5b/users/caf9d7b8de42413e9854df1b17d670ba/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.473553 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:40.488560 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128516bb-3c55-4a59-9356-cb9afb6a3dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d233074525a748a9abd2d7786bfcea5b, user_id=caf9d7b8de42413e9854df1b17d670ba, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:40.491232 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128516bb-3c55-4a59-9356-cb9afb6a3dbf admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:40.504293 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1466/2913] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:40 2022] PUT /identity/v3/projects/d233074525a748a9abd2d7786bfcea5b/users/caf9d7b8de42413e9854df1b17d670ba/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:40.508322 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae2e23-5f95-46f1-b613-3a81ea4e16c4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:40.519138 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae2e23-5f95-46f1-b613-3a81ea4e16c4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:40.575196 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b583c-f471-47f1-b269-509051886130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:40.575794 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b583c-f471-47f1-b269-509051886130 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.576101 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b583c-f471-47f1-b269-509051886130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.576379 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b583c-f471-47f1-b269-509051886130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.577015 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b583c-f471-47f1-b269-509051886130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:40.578598 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b583c-f471-47f1-b269-509051886130 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:40.589137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1448/2914] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:40.593223 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128516bb-3c55-4a59-9356-cb9afb6a3dbf admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:40.604477 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128516bb-3c55-4a59-9356-cb9afb6a3dbf admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:40.610372 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaa980-11f2-4594-88d5-b46bd30ac923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:40.611023 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa980-11f2-4594-88d5-b46bd30ac923 admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.611300 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa980-11f2-4594-88d5-b46bd30ac923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.611573 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa980-11f2-4594-88d5-b46bd30ac923 admin admin] PATH_INFO: `/v3/projects/d233074525a748a9abd2d7786bfcea5b/users/caf9d7b8de42413e9854df1b17d670ba/roles/3db28152165c45b3bc5ac611f2a8a612` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.613561 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaa980-11f2-4594-88d5-b46bd30ac923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d233074525a748a9abd2d7786bfcea5b, user_id=caf9d7b8de42413e9854df1b17d670ba, role_id=3db28152165c45b3bc5ac611f2a8a612)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:40.616246 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaa980-11f2-4594-88d5-b46bd30ac923 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:40.627638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1467/2915] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:40 2022] PUT /identity/v3/projects/d233074525a748a9abd2d7786bfcea5b/users/caf9d7b8de42413e9854df1b17d670ba/roles/3db28152165c45b3bc5ac611f2a8a612 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:40.637663 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cb36d-0d5d-4a1e-91a2-dc0adf0dd571 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.637955 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cb36d-0d5d-4a1e-91a2-dc0adf0dd571 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.638230 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cb36d-0d5d-4a1e-91a2-dc0adf0dd571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.666507 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-946cb36d-0d5d-4a1e-91a2-dc0adf0dd571 None None] MFA Rules not processed for user `caf9d7b8de42413e9854df1b17d670ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:40.668161 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-946cb36d-0d5d-4a1e-91a2-dc0adf0dd571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:40.735717 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1449/2916] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:40.746822 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaa980-11f2-4594-88d5-b46bd30ac923 admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:40.748546 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-fadaa980-11f2-4594-88d5-b46bd30ac923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:40.807148 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaa980-11f2-4594-88d5-b46bd30ac923 admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:40.810901 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c22ee4-decc-401e-a81d-a04e24cde086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:40.811509 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c22ee4-decc-401e-a81d-a04e24cde086 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:40.811834 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c22ee4-decc-401e-a81d-a04e24cde086 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:40.812159 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c22ee4-decc-401e-a81d-a04e24cde086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:40.868964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c22ee4-decc-401e-a81d-a04e24cde086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:40.870422 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c22ee4-decc-401e-a81d-a04e24cde086 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:40.880131 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1468/2917] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:44:42.379128 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cb36d-0d5d-4a1e-91a2-dc0adf0dd571 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:42.380601 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-946cb36d-0d5d-4a1e-91a2-dc0adf0dd571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:42.393097 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cb36d-0d5d-4a1e-91a2-dc0adf0dd571 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:42.396608 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a09b3-4bdb-41eb-ba8b-0ed75c63b109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:42.397193 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a09b3-4bdb-41eb-ba8b-0ed75c63b109 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:42.397471 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a09b3-4bdb-41eb-ba8b-0ed75c63b109 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:42.397826 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a09b3-4bdb-41eb-ba8b-0ed75c63b109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:42.453568 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a09b3-4bdb-41eb-ba8b-0ed75c63b109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:42.454996 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a09b3-4bdb-41eb-ba8b-0ed75c63b109 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:42.464259 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1450/2918] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:44:42.984352 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c22ee4-decc-401e-a81d-a04e24cde086 service cinder] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:42.993182 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c22ee4-decc-401e-a81d-a04e24cde086 service cinder] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:42.997474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae547a-8852-45c4-a3e9-268fd69d7f7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:42.998093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae547a-8852-45c4-a3e9-268fd69d7f7d service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:42.998478 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae547a-8852-45c4-a3e9-268fd69d7f7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:42.998751 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae547a-8852-45c4-a3e9-268fd69d7f7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:43.007679 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae547a-8852-45c4-a3e9-268fd69d7f7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:43.009187 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae547a-8852-45c4-a3e9-268fd69d7f7d service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:43.019722 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1469/2919] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:44:46.890398 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a09b3-4bdb-41eb-ba8b-0ed75c63b109 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:46.894587 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:46.956142 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a09b3-4bdb-41eb-ba8b-0ed75c63b109 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:46.960669 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:46.961280 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:46.961557 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:46.961829 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] PATH_INFO: `/v3/users/c0689d5d4c314f388f029889424b2169` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:46.962941 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0689d5d4c314f388f029889424b2169)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:46.964907 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:47.010382 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0689d5d4c314f388f029889424b2169'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.011024 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0689d5d4c314f388f029889424b2169'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.011627 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0689d5d4c314f388f029889424b2169'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.019388 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0689d5d4c314f388f029889424b2169'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.026453 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0689d5d4c314f388f029889424b2169'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.043936 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1451/2920] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:44:46 2022] DELETE /identity/v3/users/c0689d5d4c314f388f029889424b2169 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:47.048377 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26b78b-a38a-4180-886a-d076c7066c41 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:47.048836 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26b78b-a38a-4180-886a-d076c7066c41 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:47.048931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26b78b-a38a-4180-886a-d076c7066c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:47.069394 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-6d26b78b-a38a-4180-886a-d076c7066c41 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:47.070593 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6d26b78b-a38a-4180-886a-d076c7066c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:47.128204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1470/2921] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:47.137995 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:47.139415 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:47.196785 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b0273-3d8c-46c0-9b22-c52c5317c60b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:47.200493 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf1f56-2866-4ffc-84f5-282ad387e7fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:47.201090 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf1f56-2866-4ffc-84f5-282ad387e7fd service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:47.201370 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf1f56-2866-4ffc-84f5-282ad387e7fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:47.201642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf1f56-2866-4ffc-84f5-282ad387e7fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:47.210573 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf1f56-2866-4ffc-84f5-282ad387e7fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:47.211993 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf1f56-2866-4ffc-84f5-282ad387e7fd service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:47.221494 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1452/2922] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:44:47.482717 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26b78b-a38a-4180-886a-d076c7066c41 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:47.484273 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-6d26b78b-a38a-4180-886a-d076c7066c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:47.498746 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26b78b-a38a-4180-886a-d076c7066c41 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:47.503399 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:47.504036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:47.504343 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:47.504741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] PATH_INFO: `/v3/projects/556d4306d05e40508eee690bf8c3a7bb` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:47.505866 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] RBAC: Authorizing `identity:delete_project(project_id=556d4306d05e40508eee690bf8c3a7bb)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:47.507889 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:47.515113 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:47.556189 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '556d4306d05e40508eee690bf8c3a7bb', 'request_id': 'req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.562917 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '556d4306d05e40508eee690bf8c3a7bb', 'request_id': 'req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.568499 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '556d4306d05e40508eee690bf8c3a7bb', 'request_id': 'req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.581424 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] The token cache is being invalidate because project 556d4306d05e40508eee690bf8c3a7bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:44:47.581741 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.582359 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.583815 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1471/2923] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:44:47 2022] DELETE /identity/v3/projects/556d4306d05e40508eee690bf8c3a7bb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:47.587761 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf1f56-2866-4ffc-84f5-282ad387e7fd service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:47.597586 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf1f56-2866-4ffc-84f5-282ad387e7fd service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:47.619427 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:47.657788 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:47.658410 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:47.658690 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:47.658966 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] PATH_INFO: `/v3/users/31ce3776832846229b2ca3f0d23c35cf` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:47.660103 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=31ce3776832846229b2ca3f0d23c35cf)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:47.662020 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:47.706217 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31ce3776832846229b2ca3f0d23c35cf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.706863 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31ce3776832846229b2ca3f0d23c35cf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.707465 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31ce3776832846229b2ca3f0d23c35cf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.714950 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31ce3776832846229b2ca3f0d23c35cf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.721793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31ce3776832846229b2ca3f0d23c35cf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:47.739351 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1453/2924] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:44:47 2022] DELETE /identity/v3/users/31ce3776832846229b2ca3f0d23c35cf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:48.016038 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:48.018404 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:48.037217 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5b4a3-2d72-4939-a1e6-9538ea52a00b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:48.096937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:48.097633 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:48.097758 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:48.098036 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] PATH_INFO: `/v3/projects/d3d3959c58644bfe9da796e3fa2918b2` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:48.099179 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3d3959c58644bfe9da796e3fa2918b2)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:48.101165 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:48.146674 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d3959c58644bfe9da796e3fa2918b2', 'request_id': 'req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.153594 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d3959c58644bfe9da796e3fa2918b2', 'request_id': 'req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.158961 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d3959c58644bfe9da796e3fa2918b2', 'request_id': 'req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.172269 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] The token cache is being invalidate because project d3d3959c58644bfe9da796e3fa2918b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:44:48.172553 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.173168 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.174629 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1472/2925] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:44:48 2022] DELETE /identity/v3/projects/d3d3959c58644bfe9da796e3fa2918b2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:48.178666 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:48.180139 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:48.192606 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f5243-1b61-4f4d-905a-bb1b5c20c9cb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:48.247947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e740cb-1113-4ad0-b309-f16f562266b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:48.248597 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e740cb-1113-4ad0-b309-f16f562266b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:48.248877 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e740cb-1113-4ad0-b309-f16f562266b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:48.249150 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e740cb-1113-4ad0-b309-f16f562266b4 admin admin] PATH_INFO: `/v3/users/caf9d7b8de42413e9854df1b17d670ba` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:48.250260 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e740cb-1113-4ad0-b309-f16f562266b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf9d7b8de42413e9854df1b17d670ba)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:48.252199 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e740cb-1113-4ad0-b309-f16f562266b4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:48.297873 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:48.299547 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a2e740cb-1113-4ad0-b309-f16f562266b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf9d7b8de42413e9854df1b17d670ba'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.300348 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a2e740cb-1113-4ad0-b309-f16f562266b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf9d7b8de42413e9854df1b17d670ba'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.301485 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a2e740cb-1113-4ad0-b309-f16f562266b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf9d7b8de42413e9854df1b17d670ba'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.310121 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a2e740cb-1113-4ad0-b309-f16f562266b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf9d7b8de42413e9854df1b17d670ba'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.317311 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a2e740cb-1113-4ad0-b309-f16f562266b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf9d7b8de42413e9854df1b17d670ba'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.334048 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1454/2926] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:44:48 2022] DELETE /identity/v3/users/caf9d7b8de42413e9854df1b17d670ba => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:48.824153 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:48.825567 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:48.839338 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b18d7-b84b-4e1c-bbc4-78172cfd0b3b admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:48.860071 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:48.898729 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:48.899356 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:48.899638 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:48.899914 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] PATH_INFO: `/v3/projects/d233074525a748a9abd2d7786bfcea5b` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:48.901089 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d233074525a748a9abd2d7786bfcea5b)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:48.903000 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:48.957871 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd233074525a748a9abd2d7786bfcea5b', 'request_id': 'req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.964634 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd233074525a748a9abd2d7786bfcea5b', 'request_id': 'req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.971241 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd233074525a748a9abd2d7786bfcea5b', 'request_id': 'req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.983452 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] The token cache is being invalidate because project d233074525a748a9abd2d7786bfcea5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:44:48.983745 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.984398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-6334f9c4-8ef7-4079-b3cf-9fad509ac0a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:44:48.985922 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1473/2927] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:44:48 2022] DELETE /identity/v3/projects/d233074525a748a9abd2d7786bfcea5b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:48.990808 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864b848-18f2-4282-aace-e49116693b4d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:48.991101 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864b848-18f2-4282-aace-e49116693b4d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:48.991386 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864b848-18f2-4282-aace-e49116693b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:49.011551 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1864b848-18f2-4282-aace-e49116693b4d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:49.012793 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1864b848-18f2-4282-aace-e49116693b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:49.070676 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1455/2928] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:49.080692 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827552ca-d1d0-4c09-a32e-f5294945b78d None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:49.080931 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827552ca-d1d0-4c09-a32e-f5294945b78d None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:49.081203 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827552ca-d1d0-4c09-a32e-f5294945b78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:49.100853 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-827552ca-d1d0-4c09-a32e-f5294945b78d None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:49.102054 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-827552ca-d1d0-4c09-a32e-f5294945b78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:49.111619 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1474/2929] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:49.115852 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864b848-18f2-4282-aace-e49116693b4d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:49.125686 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864b848-18f2-4282-aace-e49116693b4d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:49.130305 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb2c4a-209a-4f1f-8d2a-d4983f886e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:49.130881 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb2c4a-209a-4f1f-8d2a-d4983f886e93 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:49.131159 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb2c4a-209a-4f1f-8d2a-d4983f886e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:49.131433 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb2c4a-209a-4f1f-8d2a-d4983f886e93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:49.132115 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb2c4a-209a-4f1f-8d2a-d4983f886e93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:44:49.132422 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb2c4a-209a-4f1f-8d2a-d4983f886e93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:49.134027 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb2c4a-209a-4f1f-8d2a-d4983f886e93 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:49.146028 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1456/2930] 10.222.0.86 () {58 vars in 1153 bytes} [Wed Aug 24 10:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:44:49.149937 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827552ca-d1d0-4c09-a32e-f5294945b78d None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:49.159607 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827552ca-d1d0-4c09-a32e-f5294945b78d None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:49.165077 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de584e46-ebc4-4908-9598-878218c251fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:49.166102 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de584e46-ebc4-4908-9598-878218c251fe admin admin] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:49.166384 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de584e46-ebc4-4908-9598-878218c251fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:49.166654 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de584e46-ebc4-4908-9598-878218c251fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:49.167275 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de584e46-ebc4-4908-9598-878218c251fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:49.169444 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de584e46-ebc4-4908-9598-878218c251fe admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:49.198936 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1475/2931] 10.222.0.86 () {60 vars in 1153 bytes} [Wed Aug 24 10:44:49 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:44:49.203017 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb2c4a-209a-4f1f-8d2a-d4983f886e93 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:49.212956 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb2c4a-209a-4f1f-8d2a-d4983f886e93 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:49.270607 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282463c5-45fd-4299-a065-c716e0b08cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:49.270718 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282463c5-45fd-4299-a065-c716e0b08cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:49.270959 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282463c5-45fd-4299-a065-c716e0b08cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:49.271275 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282463c5-45fd-4299-a065-c716e0b08cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:49.271994 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282463c5-45fd-4299-a065-c716e0b08cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:49.273947 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282463c5-45fd-4299-a065-c716e0b08cc5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:49.296095 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1457/2932] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Aug 24 10:44:49 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:44:49.300294 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de584e46-ebc4-4908-9598-878218c251fe admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:49.311279 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de584e46-ebc4-4908-9598-878218c251fe admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:49.315865 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954c7e2-1e48-42cf-9578-db7593e368ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:49.316489 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954c7e2-1e48-42cf-9578-db7593e368ef admin admin] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:49.316768 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954c7e2-1e48-42cf-9578-db7593e368ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:49.317041 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954c7e2-1e48-42cf-9578-db7593e368ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:49.317681 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954c7e2-1e48-42cf-9578-db7593e368ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:49.319274 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954c7e2-1e48-42cf-9578-db7593e368ef admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:49.328964 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1476/2933] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:49.332924 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282463c5-45fd-4299-a065-c716e0b08cc5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:49.342510 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282463c5-45fd-4299-a065-c716e0b08cc5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:49.347064 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48abed4-531f-4548-8990-64ca6c6ceb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:49.347642 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48abed4-531f-4548-8990-64ca6c6ceb68 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:49.347920 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48abed4-531f-4548-8990-64ca6c6ceb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:49.348219 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48abed4-531f-4548-8990-64ca6c6ceb68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:49.348853 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48abed4-531f-4548-8990-64ca6c6ceb68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:49.350413 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48abed4-531f-4548-8990-64ca6c6ceb68 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:49.361295 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1458/2934] 10.222.0.86 () {58 vars in 1122 bytes} [Wed Aug 24 10:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:44:49.365408 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954c7e2-1e48-42cf-9578-db7593e368ef admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:49.376212 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954c7e2-1e48-42cf-9578-db7593e368ef admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:49.380778 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75ec4a-fde0-4fdc-a0cc-865b0da29cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:49.381415 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75ec4a-fde0-4fdc-a0cc-865b0da29cea admin admin] REQUEST_METHOD: `PUT` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:49.381696 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75ec4a-fde0-4fdc-a0cc-865b0da29cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:49.382183 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75ec4a-fde0-4fdc-a0cc-865b0da29cea admin admin] PATH_INFO: `/v3/projects/90ba3a8a7b254290be25e4f5d5469fe1/users/9b606ff78ea04949b116ab6f9d7121af/roles/7ab2687826544b8e80c709d6061162d4` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:49.396150 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75ec4a-fde0-4fdc-a0cc-865b0da29cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ba3a8a7b254290be25e4f5d5469fe1, user_id=9b606ff78ea04949b116ab6f9d7121af, role_id=7ab2687826544b8e80c709d6061162d4)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:49.398826 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75ec4a-fde0-4fdc-a0cc-865b0da29cea admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:49.412379 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1477/2935] 10.222.0.86 () {60 vars in 1483 bytes} [Wed Aug 24 10:44:49 2022] PUT /identity/v3/projects/90ba3a8a7b254290be25e4f5d5469fe1/users/9b606ff78ea04949b116ab6f9d7121af/roles/7ab2687826544b8e80c709d6061162d4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:44:49.423222 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206012dd-b7bd-4e88-bc31-68e4df85a22e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:49.423599 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206012dd-b7bd-4e88-bc31-68e4df85a22e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:49.423874 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206012dd-b7bd-4e88-bc31-68e4df85a22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:49.452685 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-206012dd-b7bd-4e88-bc31-68e4df85a22e None None] MFA Rules not processed for user `9b606ff78ea04949b116ab6f9d7121af`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:44:49.453898 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-206012dd-b7bd-4e88-bc31-68e4df85a22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:49.470805 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:49.521013 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1459/2936] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:44:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:44:49.532204 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75ec4a-fde0-4fdc-a0cc-865b0da29cea admin admin] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:44:49.533885 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-ae75ec4a-fde0-4fdc-a0cc-865b0da29cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:44:49.589606 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75ec4a-fde0-4fdc-a0cc-865b0da29cea admin admin] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:44:49.593144 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abce0d-22ff-42cc-91f3-8e84ca272d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd7b580705f247699aeb58772e668730', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:44:49.593742 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abce0d-22ff-42cc-91f3-8e84ca272d78 service cinder] REQUEST_METHOD: `GET` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:44:49.594017 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abce0d-22ff-42cc-91f3-8e84ca272d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:44:49.594283 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abce0d-22ff-42cc-91f3-8e84ca272d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:44:49.599332 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:49.605376 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abce0d-22ff-42cc-91f3-8e84ca272d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:44:49.607037 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abce0d-22ff-42cc-91f3-8e84ca272d78 service cinder] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:44:49.617005 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1478/2937] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:44:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:44:59.518132 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=19 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:59.613905 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=3 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:45:08.893735 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206012dd-b7bd-4e88-bc31-68e4df85a22e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:45:08.954273 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206012dd-b7bd-4e88-bc31-68e4df85a22e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:45:08.958743 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:45:08.959358 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:45:08.959650 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:45:08.959911 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] PATH_INFO: `/v3/users/9b606ff78ea04949b116ab6f9d7121af` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:45:08.961061 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b606ff78ea04949b116ab6f9d7121af)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:45:08.962975 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:45:09.009253 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b606ff78ea04949b116ab6f9d7121af'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:45:09.009909 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b606ff78ea04949b116ab6f9d7121af'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:45:09.010522 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b606ff78ea04949b116ab6f9d7121af'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:45:09.017637 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b606ff78ea04949b116ab6f9d7121af'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:45:09.024614 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b606ff78ea04949b116ab6f9d7121af'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:45:09.042164 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1460/2938] 10.222.0.86 () {58 vars in 1224 bytes} [Wed Aug 24 10:45:08 2022] DELETE /identity/v3/users/9b606ff78ea04949b116ab6f9d7121af => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:45:09.047498 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575ca9d-adc9-487c-81a4-6b91d064a720 None None] REQUEST_METHOD: `POST` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:45:09.047971 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575ca9d-adc9-487c-81a4-6b91d064a720 None None] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:45:09.048093 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575ca9d-adc9-487c-81a4-6b91d064a720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:45:09.066492 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.auth.core [None req-d575ca9d-adc9-487c-81a4-6b91d064a720 None None] MFA Rules not processed for user `9757335c28f74ea5a8fbdedc0ae7682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:45:09.067692 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d575ca9d-adc9-487c-81a4-6b91d064a720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:45:09.128031 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1479/2939] 10.222.0.86 () {58 vars in 958 bytes} [Wed Aug 24 10:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:45:09.137791 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:45:09.139211 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:45:09.206293 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f245ab-8961-466c-a1a3-9c3571243288 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:45:09.210000 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964d3ca-334c-4b11-9f83-864aad5d890f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27cc091f3b094cb1a1e2650a3fbd7671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:45:09.210593 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964d3ca-334c-4b11-9f83-864aad5d890f service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:45:09.210868 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964d3ca-334c-4b11-9f83-864aad5d890f service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:45:09.211137 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964d3ca-334c-4b11-9f83-864aad5d890f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:45:09.220613 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964d3ca-334c-4b11-9f83-864aad5d890f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:45:09.222029 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964d3ca-334c-4b11-9f83-864aad5d890f service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:45:09.231771 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1461/2940] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:45:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:45:09.492457 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575ca9d-adc9-487c-81a4-6b91d064a720 None None] Authenticating user token {{(pid=81464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:45:09.494152 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.fernet_utils [None req-d575ca9d-adc9-487c-81a4-6b91d064a720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:45:09.506257 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575ca9d-adc9-487c-81a4-6b91d064a720 None None] Validating token access rules against request {{(pid=81464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:45:09.510875 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9757335c28f74ea5a8fbdedc0ae7682a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f71a167399734714a60b777eea4e2ecc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:45:09.511991 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:45:09.512305 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:45:09.512578 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] PATH_INFO: `/v3/projects/90ba3a8a7b254290be25e4f5d5469fe1` {{(pid=81464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:45:09.513704 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90ba3a8a7b254290be25e4f5d5469fe1)` {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:45:09.515644 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] RBAC: Authorization granted {{(pid=81464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:45:09.562805 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ba3a8a7b254290be25e4f5d5469fe1', 'request_id': 'req-9df830ed-1581-466a-82cc-69965d3a5ec6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:45:09.568942 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ba3a8a7b254290be25e4f5d5469fe1', 'request_id': 'req-9df830ed-1581-466a-82cc-69965d3a5ec6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:45:09.575385 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ba3a8a7b254290be25e4f5d5469fe1', 'request_id': 'req-9df830ed-1581-466a-82cc-69965d3a5ec6'} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:45:09.587443 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] The token cache is being invalidate because project 90ba3a8a7b254290be25e4f5d5469fe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:45:09.587731 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:45:09.588398 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG keystone.notifications [None req-9df830ed-1581-466a-82cc-69965d3a5ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:45:09.590550 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: [pid: 81464|app: 0|req: 1480/2941] 10.222.0.86 () {58 vars in 1233 bytes} [Wed Aug 24 10:45:09 2022] DELETE /identity/v3/projects/90ba3a8a7b254290be25e4f5d5469fe1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:45:19.227133 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:45:19.584474 dv-cinder-854270-2-iscsi devstack@keystone.service[81464]: DEBUG dbcounter [-] [81464] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:47:45.898984 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964d3ca-334c-4b11-9f83-864aad5d890f service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:47:45.900454 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2964d3ca-334c-4b11-9f83-864aad5d890f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:47:45.971040 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964d3ca-334c-4b11-9f83-864aad5d890f service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:47:46.029146 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413dfd2-3325-4506-bfac-4e92d52fe0ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8d30e87c624d6180656d998afcebe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3955b2c7daac4b47a5a38418ec79a089', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:47:46.029668 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413dfd2-3325-4506-bfac-4e92d52fe0ce service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:47:46.029950 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413dfd2-3325-4506-bfac-4e92d52fe0ce service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:47:46.030226 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413dfd2-3325-4506-bfac-4e92d52fe0ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:47:46.033116 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f413dfd2-3325-4506-bfac-4e92d52fe0ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:47:46.043836 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413dfd2-3325-4506-bfac-4e92d52fe0ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:47:46.045694 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413dfd2-3325-4506-bfac-4e92d52fe0ce service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:47:46.055569 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1462/2942] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Aug 24 10:47:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:47:56.051289 dv-cinder-854270-2-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=58 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}